Methods and apparatus for communicating geolocated data

ABSTRACT

Geolocated information is communicated to a user based upon a position of smart device in a building as determined by optical recognition of a first visual identifier, a second visual identifier and a third visual identifier. A distance determined from each of the visual identifiers, as well as a direction of interest indicated by a user. A user interface is generated for display on a Smart Device based upon the position of the Smart Device and direction of interest.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to the Non-Provisional patentapplication Ser. No. 17/409,919 filed Aug. 24, 2021 and entitled METHODSOF COMMUNICATING GEOLOCATED DATA BASED UPON A SELF-VERIFYING ARRAY OFNODES as a continuation, which in turn claims priority toNon-Provisional patent application Ser. No. 17/176,849 filed Feb. 16,2021 and entitled METHOD OF WIRELESS GEOLOCATED INFORMATIONCOMMUNICATION IN SELF-VERIFYING ARRAYS as a continuation, which in turnclaims priority to the Non-Provisional patent application Ser. No.16/915,155 filed Jun. 29, 2020 and entitled METHOD OF WIRELESSDETERMINATION OF A POSITION OF A NODE as a continuation, which in turnclaims priority to the Non-Provisional patent application Ser. No.16/775,223 filed Jan. 28, 2020 and entitled SPATIAL SELF-VERIFYING ARRAYOF NODES as a continuation, which in turn claims priority to the NonProvisional patent application Ser. No. 16/528,104, filed Jul. 31, 2019and entitled SMART CONSTRUCTION WITH AUTOMATED DETECTION OF ADVERSESTRUCTURE CONDITIONS AND REMEDIATION as a continuation; and to theNon-Provisional U.S. patent application Ser. No. 16/504,919, filed Jul.8, 2019 and entitled METHOD AND APPARATUS FOR POSITION BASED QUERY WITHAUGMENTED REALITY HEADGEAR as a continuation, which in turn claimspriority to Provisional Patent Application Ser. No. 62/793,714, filedJan. 17, 2018 and entitled METHOD AND APPARATUS FOR ORIENTEERING WITHAUGMENTED REALITY HEADGEAR; and as a continuation application to theNon-Provisional U.S. patent application Ser. No. 16/657,660, filed Oct.18, 2019 and entitled METHOD AND APPARATUS FOR CONSTRUCTION ANDOPERATION OF CONNECTED INFRASTRUCTURE; and as a continuation in partapplication to the Non-Provisional U.S. patent application Ser. No.16/688,775, filed Nov. 19, 2019 and entitled METHOD AND APPARATUS FORWIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICE;and as a continuation in part application to the Non-Provisional U.S.patent application Ser. No. 16/721,906, filed Dec. 19, 2019 and entitledMETHOD AND APPARATUS FOR WIRELESS DETERMINATION OF POSITION ANDORIENTATION OF A SMART DEVICE. The contents of each of the heretoforeclaimed matters are relied upon and incorporated herein by reference.

The present application also references the following relatedapplications whose content are relied upon and incorporated herein byreference; the Non Provisional patent application Ser. No. 16/549,503,filed Aug. 23, 2019 and entitled METHOD AND APPARATUS FOR AUGMENTEDVIRTUAL MODELS AND ORIENTEERING; and the Non Provisional patentapplication Ser. No. 15/703,310, filed Sep. 13, 2017 and entitledBUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVEPERFORMANCE TRACKING; and the Non Provisional patent application Ser.No. 16/161,823, filed Oct. 16, 2018 and entitled BUILDING MODEL WITHCAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATA; and the NonProvisional patent application Ser. No. 15/887,637, filed Feb. 2, 2018and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES ANDEXPERIENTIAL DATA; and the Non Provisional patent application Ser. No.16/165,517, filed Oct. 19, 2018 and entitled BUILDING VITAL CONDITIONSMONITORING,

FIELD OF THE INVENTION

The present invention relates to methods and apparatus for aself-verifying array of Nodes, a spatial position of individual is Nodesis verified using values for communication variables generated bymultiple communications involving multiple Nodes. More specifically, thepresent invention provides apparatus and methods for designating aposition of a Node that includes a wireless transceiver relative to abase position or relative to another Node based upon wirelesscommunications amongst multiple wireless transceivers included in anarray.

BACKGROUND OF THE INVENTION

Wireless determination of a position has been known for many years. Eachtechnique and corresponding wavelength has its strengths and drawbacks.One significant drawback has been the specialized equipment and trainingrequired to utilize wireless position determination equipment. Forexample, use of systems such as radar and lidar requires specializedequipment and training.

In contrast, most people have access to a smart device. Theproliferation of global positioning system (GPS) capabilities by smartdevices has alleviated the need for such specialized equipment andtraining by incorporating the specialized circuitry into the smartdevice, and proliferating apps that operate the GPS circuitry. Smartdevices are used almost ubiquitously by people in first-world populationcenters. Smart phones and tablets are often within reach of peoplecapable of operating them and are relied upon for almost any purpose forwhich an app may be written.

Some Smart Devices have been incorporated into devices normally operatedby people in their daily lives. Such Smart Devices may be included in alarger “smart” ecosystem, a popular example of which is a smart home.The term “smart home” generally refers to use of appliances, securitydevices and climate control devices that are controlled by a processorof one sort or another. In general, these are consumer devices, made forthe convenience of the user and efficiency of use of a related device.

However, known geolocation technologies (as may be deployed with modernSmart Devices) also have drawbacks. GPS is purposefully limited in itsaccuracy by the government. Other technologies and correspondingstandards (which operate at different wavelength bands), such asBluetooth and WiFi, are easily obstructed and have very limited range.

Many devices using the Internet of Things include communication elementsdeploying any one of numerous standards such as Bluetooth, WiFi,Cellular and other examples. It would be useful to improvecommunications, location tracking, and generalized tracking of devicescarrying wireless communications especially in complex environments thatare larger than the typical broadcast range of a particular wavelengthor have aspects that inhibit wireless communication.

SUMMARY OF THE INVENTION

Accordingly, the present invention provides for a self-verifying arrayof Nodes (sometimes referred to herein as “SVAN”) that verifiespositions of respective Nodes included in the array. A position for eachNode is generated based upon sets of values of variables derived fromwireless communications (sometimes referred to herein as “positiondetermination variables”). The position determination variables mayinclude one or more of: a time of transmission of a data set duringwireless communication between Nodes; a time of arrival of a data setduring wireless communication between Nodes; a phase change betweendisparate antennas receiving a wireless communication; an angle ofarrival of a data set; an angle of departure of a data set; a quality ofa wireless transmission (e.g., based on a presence of noise in thereceived transmission); a strength of a wireless transmission (e.g., asmeasured by amplitude of a received transmission); or other factorinfluencing a wireless data transmission.

According to the present invention, an array self-verifies positions ofrespective Nodes included in the array by generating multiple sets ofvalues for position determination variables for each of the Nodes in thearray. Each set of values is based upon multiple disparatecommunications involving respective pairs of a transmitting Node and areceiving Node. In this manner, multiple sets of values for positiondetermination variables are generated for each respective Node during agiven timeframe. Each set of values for position determination variablesmay be used to verify a position of a designated Node by comparing aposition determined via use of a first set of values for variables topositions determined via use of set(s) of variables other than the firstset of variables. Each determined position for a given Node therebyverifies or challenges other determined positions.

In some embodiments, outlier sets of position determination variablesmay be excluded. In another aspect, in some embodiments, an algorithmmay be used to generate a composite position for a given Node based uponmultiple sets of position determination variables (for example, bygenerating a weighted average of expected positions based on thedisparate sets of values of position determination variables).

In various embodiments, a determined position of Node may include aposition for a first Node relative to a position of a second Node, orrelative to a base position. Each position is generally represented as aset of coordinates. The coordinates may include, for example, Cartesiancoordinates, cylindrical coordinates and/or Polar coordinates. The Nodesmay include transceivers transmitting and receiving one designatedbandwidth of communication wavelength; or transceivers operatingaccording to disparate wireless protocols and across multiplebandwidths. In some embodiments, a wireless communications Node mayreceive a data set via a first wavelength (and first associatedprotocol) and transmit some or all of the data set via a secondwavelength (and second associated protocol) or combine transceiverscapable of communicating via multiple wavelengths and protocols.

The self-verifying arrays of Nodes of the present invention includecollections of numerous wireless communication Nodes operative tocooperatively enhance communications, location tracking, anddetermination of other useful aspects of an array of Nodes, such asproximity of Nodes to each other and/or an item of interest, distance ofNodes to each other and/or an item of interest, direction of Nodes toeach other and/or an item of interest, and whether or not two Nodes arecapable of direct communication between themselves. Self-verifyingarrays may be deployed to significantly optimize and improve accuracy ofdetermining a location of a Sensor, tracking of items, tracking ofAgents and/or persons in real-world scenarios, such as a constructionenvironment, a parking area, a health care Structure, a hotel, aconvention center, a recreation area, an amusement park, and a partiallybuilt or completed Structure, as non-limiting examples.

In some exemplary embodiments, a designated location may includestationary wireless Nodes that are fixed to stationary item in orproximate to a Structure such as a building part or a stanchion securedto a ground point. The site may also include mobile wireless Nodes whichmay be fixed to items, persons and/or Agents capable of attainingdynamic locations. Various assets and building materials may be fittedwith wireless Nodes that are combined into a self-verifying array.

In some examples, wireless communications between wireless Nodes may beaccomplished in adherence to a Bluetooth protocol, such as, by ways ofnon-limiting example, Bluetooth 5.1 or Bluetooth Low Energy (BLE 5.1).In other examples, RFID type tags may communicate information inresponse to a stimulus. In some examples, energy to power a Node may beprovided by a wireless transmission to the Node to be powered.

In general, Nodes making up a self-verifying array communicate to otherwireless Nodes. The wireless communications may include one or both ofsensor data and location-identification data. Location-identificationdata may include one or more of: values for variables that are usefulfor determining a position, information useful for determine a polarcoordinate (e.g., angle of arrival; angle of departure; and distance);or information useful for determining a Cartesian Coordinate (e.g.,X,Y,Z coordinates). The location-identification data may be one or bothof: relative to two Nodes or relative to a base position. By way ofnon-limiting example, of location-identification data may include one ormore of: transmitting and receiving timing data; angle of arrival; angleof departure; a calculated distance; and a set of coordinates.

Location-identification data involving a particular Node may begenerated by that Node. For example, a Node X may generatelocation-identification data relative to multiple other Nodes with whichNode X is capable of communicating. Node X may also receive, viawireless communication, location data generated by other Nodes. Node Xmay aggregate both types of location-identification data and transmitthe aggregated data out to any wireless transceiver within range that iscapable of receiving the aggregated data.

A controller, such as a controller in a Smart Device or in a cloudserver, may generate a map indicating locations of various Nodes at aninstance in time. Each location of each Node may be based upon one ormore sets of location-identification information since each Node maycommunicate with multiple other Nodes. Accuracy of a location of aparticular Node may be enhanced by mathematically blending multiple setsof location-identification information for that particular Node, such asan average of reported data (including, in some embodiments, a weightedaverage). In some embodiments, a strength of a wireless communicationmay be determined and recorded and considered in the blending ofmultiple sets of location-identification data.

In some embodiments, a sensor may be co-located with a particular Node.In this manner, data generated by the Sensor during a particular timeperiod may be associated with the position of a co-located Node. In someembodiments, data generated by a Sensor (Sensor Data) may be transmittedbetween Nodes on a periodic basis. Transmission of Sensor Data betweenNodes may be in addition to location-identification information, orindependent of other transmissions, including transmissions oflocation-identification information. Other embodiments includetransmission of Sensor data in response to a command requesting theSensor data.

Accordingly, some embodiments include transmission and receipt of SensorData for the purposes of aggregating and retransmitting the receiveddata. The sensor data may quantify a condition at a location orproximate to a location of a Node in logical communication with theSensor. In another aspect, disparate Nodes may transmit data to otherNodes, wherein the data has been provided by Sensors co-located with orproximate to respective Nodes. Each Node may aggregate data received viacommunications with other Nodes and/or received from Sensors (orassemblies of multiple Sensors) co-located with or proximate torespective Nodes.

Sensor data may thereby be aggregated from disparate Nodes at disparatelocations across a large area occupied by Nodes that interconnect into aself-verifying array. This may be beneficial in embodiments in which thebreadth of a physical area covered by a self-verifying array of Nodesexceeds the point-to-point communication range of one or more of theNodes (e.g., based on the communication modality of the Node).

In addition, a self-verifying array may combine communications usingdisparate bandwidths and protocols to achieve superior performance in avariety of ways, such as improving communications distance, accuracy,and obstacle-penetration efficacy. A self-verifying array may alsoinclude hardwired segments to further achieve improved performance andconnectivity to resources external to the self-verifying array.

By using the self-verifying array of Nodes to effectively expand therange of an individual Node, communication and data retrieval across alarge space is improved. Specifically, establishment of a self-verifyingarray that allows for communication pathways to be established that arelonger than a range of an individual wireless access Node and verifies alocation of a communication commencement and destination, enablessuperior communications across large areas, such as a construction site,a large building, a parking area/garage, an amusement park, a publicarea, or other defined area.

For example, a large construction site may include stationary Nodes thatform a self-verifying array of Nodes over a large spatial area includedin the construction site (or even the entire construction site). Duringconstruction, a line-of-sight path between a particular wirelesscommunication base Node and a deployed Node initially interacting withthat wireless communication base Node may be cut off by variousimpediments to wireless transmission, such as equipment used to buildthe Structure, materials being stored to build the Structure, or theStructure itself. The self-verifying array may create paths thatcooperatively allow the deployed Node to connect with multiple differentwireless communication Nodes included in a self-verifying array to senda communication around the impediment to wireless communication andreach the base Node.

A mobile wireless Node included in the self-verifying array may providedynamic location aspects to the self-verifying array. Devices andmethodology allow for mobile Nodes to supplement stationary Nodes andimprove communications aspects in numerous ways. The mobile Node maytemporarily create a shorter path for communications, which may improveenergy storage aspects of a device interacting with the Node, animproved signal-to-noise aspect, or other advantages.

A communication area covered by an aggregate of wireless Nodes mayextend to a perimeter defined by communication coverage of an aggregateof the Nodes and may encompass communication obstructions within thecommunications area, wherein the obstructions are circumvented bystrategically located Nodes that communicate around the obstruction.

In still further aspects, a self-verifying array of Nodes may allow anAugmented Virtual Model (AVM) of a Structure and/or a building site tobe updated with the locations of an Agent or equipment that isco-located with a wireless Node. Similarly, building materials that areco-located with wireless Nodes may have their location determined and/orconditions experienced by the materials quantified via Sensor readings.This may occur as the materials reside in a storage location and/or asthe materials are assembled into a Structure.

Locations of personnel tagged with a wireless Node may also beidentified for logistics, safety, and other purposes. For example, insome embodiments, a Smart Device may serve as a dynamic Node. The SmartDevice may be supported by an Agent. In such embodiments, locationinformation and other Sensor data from the Smart Device may betransmitted across the self-verifying array. Accordingly, it may bepossible to track the Agent's position, biometrics, and other safetyquantities across the self-verifying array.

A mobile Agent equipped with Node with wireless communicationscapability may also transmit energy beacons into the regions that theAgent moves into. The energy beacons may energize ultralow-energyBluetooth-equipped devices, RFID tags, and the like. Thus, Nodes thathave little or no substantial battery capability may be energized andmay respond to the energization by transmitting and/or receiving datatransmissions to/from one or both of the mobile Agent and other Nodes.Transmitted data may include an identification of respective Nodes,Sensor-related information, and the like. Other protocols such asstepped power levels in transmission may supplement a location with arelative distance between the tag and the mobile agent being determined.Since the mobile Agent can perform this measurement from numerouspoints, triangulation may be used to improve the accuracy of relativelocation determination of such tags.

DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate several embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention:

FIG. 1A illustrates a block diagram of inter-relating functions includedin automated systems according to the present invention.

FIG. 1B illustrates geolocation aspects that may be used to identify aProperty and corresponding wireless modalities that may be used.

FIG. 1C illustrates a block diagram of ongoing data capture via SmartDevices and Sensors and support for predictive modeling based upon thesmart data capture.

FIG. 1D illustrates an exemplary Structure layout with various itemsdelineated in a top-down representation, according to some embodimentsof the present invention.

FIG. 1E illustrates a diagram of an Agent and Directional Image Data.

FIG. 2 illustrates a block diagram of an Augmented Virtual Modelingsystem.

FIGS. 3A-3D illustrate exemplary aspects of collecting and displayingdata generated within a Structure.

FIG. 3E illustrates the installation of wireless Nodes in a Structure.

FIG. 3F illustrates an example of deployed wireless Nodes interactingwith an Agent in proximity to a door.

FIG. 4 illustrates a Node with wireless Transceivers useful for locationdetermination and data transceiving.

FIG. 5 illustrates Reference Point Transceivers useful for locationdetermination and data transceiving.

FIG. 6 illustrates apparatus that may be used to implement aspects ofthe present invention including executable software.

FIG. 6A illustrates an exemplary block diagram of a controller withangle of arrival and angle of departure functionality.

FIG. 6B illustrates exemplary block diagram of an assembly with multipleantenna arrays such as a “puck”

FIG. 6C illustrates another view of a puck with directional antennaarrays.

FIG. 7 illustrates an exemplary mobile Smart Device that may be used toimplement aspects of the present invention including executablesoftware.

FIGS. 8, 8A-8D illustrate a device and Vectors according to variousembodiments of the present invention.

FIGS. 9A-D illustrate exemplary antenna array design examples.

FIGS. 10A-C illustrate location determination with exemplary antennaarrays.

FIG. 11 illustrates a flow chart of method steps that may be executed insome implementations of the present invention.

FIG. 11A illustrates a user entering a room with a Smart Device, whereinthe room includes various and numerous wireless communicating devices.

FIG. 11B illustrates a map displayed on a Smart Device with CartesianCoordinates as well as Polar Coordinates displayed.

FIG. 12A illustrates a physical location with various stationary andmovable wireless Nodes including camera equipped devices.

FIG. 12B illustrates a view on a Smart Device incorporating a cameravideo display with superimposed wireless device location.

FIG. 12C illustrates a view on a Smart Device showing a map of knowndevice locations as well as known user location and user orientation.

FIG. 12D illustrates a view of devices displayed on a Smart Deviceshowing a map of known location equipped SVAN Node locations as well asregionally associated non-location equipped devices.

FIG. 12E illustrates a view of a SVAN displayed on a Smart Deviceshowing movement of known location Node locations as well as movement ofregionally associated non-location equipped devices.

FIG. 12F illustrates a view of a SVAN being used to look aroundblockages.

FIG. 13A illustrates a set of mobile Nodes represented in polarcoordinates.

FIG. 13B illustrates an ability of a set of mobile Nodes to cooperatearound blocked transmission zones with line of sight.

FIG. 14 depicts methodology related to the present invention.

FIG. 15 illustrates exemplary methods of computing the distance betweentwo Nodes not having line-of-sight communications between each other.

FIG. 16 illustrates methodology related to SVAN arrays deployed withnodes on, in or associated with vehicles.

FIG. 17 illustrates methodology related to SVAN arrays associated withAgents, materials, equipment, and structural aspects.

FIG. 18 illustrates method steps of some implementations of the presentinvention relating to occupancy of space.

FIG. 19 illustrates method steps of some implementations of the presentinvention relating to geolocated communication of information.

DETAILED DESCRIPTION

The present invention provides for a spatially self-verifying array ofNodes. Specifically, Nodes may include devices capable of wirelesscommunication transmission in logical communication with a processor anda digital storage. A position for each Node may be generated based uponvalues for position determination variables. By comparing the values forposition determination variables between a single Node and multipledisparate Nodes, a position of respective Nodes in the array may bedetermined and verified.

In some embodiments, Nodes are co-located with Sensors to quantifyconditions within or proximate to Structures. Such Structures use Sensorgroups periodically and/or continuously quantify and transmit a currentcondition of the Structure. Sensor readings may be associated with atime index.

Various embodiments include methods and apparatus for construction,Deployment and maintenance of a Structure with Intelligent Automation(device, system, machine or equipment item) engaged in logical processesand Structural Messaging to communicate conditions within or proximateto the Structure. Structural Messaging includes logical communicationsgenerated by the Intelligent Automation (such as a Sensor or machine)incorporated into, affixed to, or operated within or proximate to aStructure.

In some aspects, a Sensor cluster (or a Sensor gateway, which may be aSensor cluster connected to a communications array) may be embedded intoa wall or other surface, such as an architectural aspect (e.g., abaseboard). The Sensors may be capable of quantifying a condition bygenerating a digital value based upon an environment in which the Sensoris placed. For example, the Sensors may detect vibration patterns,chemicals, temperatures, water, light waves or other indicia of acondition present. A remedial action device may, based upon a readingfrom the Sensors, be actuated in response to a quantified condition.

In general, various embodiments of the present invention enable aStructure, such as a building or infrastructure, to be active as opposedto the former passive state. The active state enables the Structure togenerate data descriptive of one or more of: a condition within aStructure; a condition proximate to the Structure; and an eventexperienced by the Structure; and in some embodiments an active stateStructure is enabled to execute an action via automation based upon aStructural Message. The action based upon a Structural Message may beexecuted independent of a user intervention, or based upon approval of auser, such as via an app on a Smart Device.

The present invention references prior applications and issued patentsowned by the applicant relating to automated apparatus and methods forgenerating improved Augmented Virtual Models (sometimes referred toherein as an “AVM”) of a Structure.

The AVM of the Property may include a conceptual model and progressthrough one or more of: a) a design stage; b) a build stage; c) aDeployment stage; d) a service stage; e) a modification stage; and f) adispensing stage. As discussed more fully herein, an AVM according tothe present invention include original design data matched to As Builtdata captured via highly accurate geolocation, direction and elevationdetermination. As Built data is matched with a time and date of dataacquisition and presented in two-dimensional (2D) and three-dimensional(3D) visual representations of the Property. The augmented modelsadditionally include data relating to features specified in a Propertydesign and data collected during building, Deployment, maintenance andmodifications to the Property. In some embodiments, a fourth dimensionof time may also be included.

An AVM includes a three- or four-dimensional model in a virtualenvironment that exists parallel to physical embodiments modeled in theAugmented Virtual Model. Details of one or more physical Structures andother features within a real estate parcel are generated and quantifiedand represented in the Augmented Virtual Model. The AVM exists inparallel to a physical Structure in that the AVM includes virtualrepresentations of physical Structures and additionally receives andaggregates data relevant to the Structures over time. The aggregation ofdata may be one or more of: a) according to an episode (e.g., onsiteinspection, repair, improvement etc.); b) periodic; and c) in real tune(without built in delay).

The experience of the physical Structure is duplicated in the virtualAugmented Virtual Model. The AVM may commence via an electronic modelgenerated via traditional CAD software or other design type software. Inaddition, the AVM may be based upon values for variables, including oneor more of: usage of a Structure; usage of components within theStructure; environmental factors encountered during a build stage orDeployment stage; and metrics related to Performance of the Structure.The metrics may be determined, for example, via measurements performedby Sensors located in and proximate to Structures located on theProperty.

In some embodiments, a technical library specific to a particularProperty and location within the Property may be maintained for eachProperty and made accessible to an onsite technician and/or remoteexpert. The library may include but is not limited to detailsdescriptive of: a Structure design, utilities, architectural andstructural history, equipment/machinery manuals; repair bulletins, andrepair/maintenance. Appropriate how-to videos may also be made availablebased upon an AVM with As Built and Experiential Data.

In another aspect, a parts ordering function may be included in theAugmented Virtual Model. Augmented parts ordering may allow a technicianto view an ordered part and view a virtual demonstration of the part inuse and procedures for replacing the part.

Aspects of the AVM may be presented via a user interface that maydisplay on a tablet or other flat screen, or in some embodiments bepresented in a virtual reality environment, such as via a virtualreality headset.

Some exemplary embodiments may include updates to an AVM that includechanges to: items or persons within the Structure, architectural orstructural aspects; time and date notation of a change in locationspecific data; a location of an item or person updated according tocoordinates such as X,Y,Z and distance data and/or an angle and distancedata (or other information pertinent to a chosen coordinate system); X,Ydata may include high level location designation within the streetaddress via triangulation (e.g., a street address) and highly specificposition designation (e.g., particular room and wall); combination oftwo types of position data; GPS, Differential GPS; references usedduring triangulation; aggregate data across multiple Structures forreference; designs that perform well; designs that fail; popularity ofvarious aspects; access to and/or generation of, multiple AugmentedVirtual Models.

In some preferred embodiments, the geographic location will be providedwith accurately placed location reference points. The location referencepoints may be accessed during activities on a Property within or closeto a Structure. (While accuracy may degrade outside the Property, thelocation reference points maintain accuracy within the Property.)

Preferred embodiments may also include reference points accuratelyplaced within a Structure located on the Property. As further discussedbelow, the reference points may include, by way of non-limiting example,a wireless transmission data transmitter operative to transmit anidentifier and location data; a visual identifier, such as a hash code,bar code, color code or the like; an infrared transmitter; a reflectivesurface, such as a mirror; or other means capable of providing areference point to be utilized in a triangulation process thatcalculates a precise location within the Structure or other Structure.

Highly accurate location position may be determined via automatedapparatus and multiple levels of increasingly accurate locationdetermination. A first level may include use of a GPS device providing areading to first identify a Property. A second level may use positiontransmitters located within, or proximate to, the Property to executetriangulation processes in view of on-site location references. A GPSlocation may additionally be associated with a high-level generaldescription of a Property, such as, one or more of: an address, a unitnumber, a lot number, a tax map number, a county designation, Plattenumber or other designator. On-site location references may include oneor more of: near field radio communication beacons at known X-Y positionreference points; line of sight with physical reference markers; codedvia ID such as bar code, hash tag, and alphanumeric or other identifier.In some embodiments, triangulation may calculate a position within aboundary created by the reference points to within millimeter range. Insome embodiments, Differential GPS may be used to accurately determine alocation of a Smart Device with a sub centimeter accuracy.

In addition to a position determination, such as latitude and longitude,or other Cartesian Coordinate (which may sometimes be indicated as an“X” or “Y” coordinate), Polar Coordinate, or GPS coordinate, the presentinvention provides for a direction (sometimes referred to herein as a“Z” direction and elevation or “r”) of a feature for which As Built datais captured and imported into the AVM.

According to the present invention, a direction dimension may be basedupon a movement of a device. For example, a device with a controller andan accelerometer, such as mobile Smart Device, may include a userdisplay that allows a direction to be indicated by movement of thedevice from a determined location acting as a base position towards anAs Built feature in an extended position. In some implementations, theSmart Device may first determine a first position based upontriangulation with the reference points and a second position (extendedposition) also based upon triangulation with the reference points. Theprocess of determination of a position based upon triangulation with thereference points may be accomplished, for example via executablesoftware interacting with the controller in the Smart Device, such as,for example via running an app on the Smart Device.

In combination with, or in place of directional movement of a deviceutilized to quantify a direction of interest to a user, some embodimentsmay include an electronic and/or magnetic Directional Indicator that maybe aligned by a user in a direction of interest. Alignment may include,for example, pointing a specified side of a device, or pointing an arrowor other symbol displayed upon a user interface on the device towards adirection of interest.

In a similar fashion, triangulation may be utilized to determine arelative elevation of the Smart Device as compared to a referenceelevation of the reference points.

It should be noted that although a Smart Device is generally operated bya human user, some embodiments of the present invention include acontroller, accelerometer, data storage medium, Image Capture Device,such as a Charge-Coupled Device (“CCD”) capture device or an infraredcapture device being available in a handheld or unmanned vehicle orother Agent.

An unmanned vehicle may include for example, an unmanned aerial vehicle(“UAV”) or ground level unit, such as a unit with wheels or tracks formobility and a radio control unit for communication.

In some embodiments, multiple unmanned vehicles may capture data in asynchronized fashion to add depth to the image capture and/or athree-dimensional and four-dimensional (over time) aspect to thecaptured data. In some implementations, UAV position will be containedwithin a perimeter and the perimeter will have multiple reference pointsto help each UAV (or other unmanned vehicle) determine a position inrelation to static features of a building within which it is operatingand also in relation to other unmanned vehicles. Still other aspectsinclude unmanned vehicles that may not only capture data but alsofunction to perform a task, such as paint a wall, drill a hole, cutalong a defined path, or other function. As stated throughout thisdisclosure, the captured data may be incorporated into the virtual modelof a Structure.

In another aspect, captured data may be compared to a library of storeddata using image recognition software to ascertain and/or affirm aspecific location, elevation and direction of an image capture locationand proper alignment with the virtual model. Still other aspects mayinclude the use of a compass incorporated into a Smart Device.

In still other implementations, a line of sight from a Smart Device,whether user operated or deployed in an unmanned vehicle, may be used toalign the Smart Device with physical reference markers and therebydetermine an X,Y position as well as a Z position. Electronic altitudemeasurement may also be used in place of, or to supplement, a knownaltitude of a nearby reference point. This may be particularly useful inthe case of availability of only a single reference point.

Reference points may be coded via identifiers, such as a UUID(Universally Unique Identifier), or other identification vehicle. Visualidentifiers may include a bar code, hash tag, Alphanumeric or othersymbol. Three-dimensional markers may also be utilized.

By way of non-limiting example, on site data capture may includedesignation of an X,Y,Z reference position and one or more of: imagecapture; infrared capture; temperature; humidity; airflow;pressure/tension; electromagnetic reading; radiation reading; soundreadings (e.g., level of noise, sound pattern to ascertain equipmentrunning and/or state of disrepair), and other vibration or Sensorreadings (such as an accelerometer or transducer).

In some embodiments, vibration data may be used to profile use of theStructure and/or equipment and machinery associated with the Structure.For example, vibration detection may be used to deter mine a presence ofa person or vehicle, a type of activity taking place; machine operation,including automated determination between proper operation of a piece ofequipment and/or machinery and faulty operation of the equipment and/ormachinery.

Glossary

“Agent” as used herein refers to a person or automation capable ofsupporting a Smart Device at a geospatial location relative to a GroundPlane.

“Ambient Data” as used herein refers to data and data streams capturedin an environment proximate to a Vantage Point and/or an equipment itemthat are not audio data or video data. Examples of Ambient Data include,but are not limited to, Sensor perception of: temperature, humidity,particulate, chemical presence, gas presence, light, electromagneticradiation, electrical power, Moisture and mineral presence.

“Analog Sensor” and “Digital Sensor” as used herein include a Sensoroperative to quantify a state in the physical world in an analog ordigital representation, respectively.

“As Built” as used herein refers to details of a physical Structureassociated with a specific location within the physical Structure orparcel and empirical data captured in relation to the specific location.

“As Built Features” as used herein refers to a feature in a virtualmodel or AVM that is based at least M part upon empirical data capturedat or proximate to a correlating physical location of the feature.Examples of As Built Features include placement of structural componentssuch as a wall, doorway, window, plumbing, electrical utility, machineryand/or improvements to a parcel, such as a well, septic, electric orwater utility line, easement, berm, pond, wet land, retaining wall,driveway, right of way and the like.

“As Built Imagery” (Image Data) as used herein means image datagenerated based upon a physical aspect.

“Augmented Virtual Model” (sometimes referred to herein as “AVM”) asused herein means a digital representation of a real Property parcelincluding one or more three-dimensional representations of physicalStructures suitable for use and As Built data captured that isdescriptive of the real Property parcel. An AVM includes As BuiltFeatures of the Structure and may include improvements and featurescontained within a Structure.

“Bluetooth” as used herein means the Wireless Personal Area Network(WPAN) standards managed and maintained by Bluetooth Special InterestGroup (SIG). Unless otherwise specifically limited to a subset of allBluetooth standards, the Bluetooth will encompass all Bluetoothstandards (including, without limitation, Bluetooth 4.0; 5.0; 5.1 andBLE versions).

“Deployment” as used herein means the placement into operation of one ormore of: a Structure machinery and an equipment item.

“Deployment Performance” as used herein means one or both of: objectiveand subjective quantification of how one or more of: Structure,machinery and an equipment item operated, which may be depicted in anAVM.

“Design Feature” as used herein, means a value for a variabledescriptive of a specific portion of a Property. A Design Feature mayinclude, for example, a size and shape of a structural element or otheraspect, such as a doorway, window, or beam; a material to be used; anelectrical service; a plumbing aspect; a data service; placement ofelectrical and data outlets; a distance, a length, a number of steps; anincline; or other discernable value for a variable associated with aStructure or Property feature.

“Digital Sensor” as used herein includes a Sensor operative to quantifya state in the physical world in a digital representation.

“Directional Indicator” as used herein means a quantification of adirection generated via one or both of: analog and digital indications.

“Experiential Data” as used herein means data captured on or proximateto a subject Structure, such data descriptive of a condition realized bythe Structure. Experiential Data is generated by one or more of: Digitaland/or Analog Sensors, transducers, Image Capture Devices, microphones,accelerometers, compasses and the like.

“Experiential Sensor Reading” as used herein means a value of a Sensoroutput generated within or proximate to a subject Structure, such outputdescriptive of a condition realized by the Structure. An ExperientialSensor Reading may be generated by one or more of: digital and/or AnalogSensors, transducers, Image Capture Devices, microphones,accelerometers, compasses and the like.

“Ground Plane” as used herein refers to a locally horizontal (or nearlyhorizontal) plane from which a direction of interest may be projected.An example of a Ground Plane is a floor of a Structure.

“Image Capture Device” or “Scanner” as used herein refers to apparatusfor capturing digital or analog image data. An Image Capture Device maybe one or both of: a two-dimensional camera or a three-dimensionalcamera. In some examples an Image Capture Device includes acharge-coupled device (“CCD”) camera.

“Intelligent Automation” as used herein refers to a logical processingby a device, system, machine or equipment item (such as data gathering,analysis, artificial intelligence, and functional operation) andcommunication capabilities.

“Moisture” as used herein means a quantity of water, which may also meana quantity of water relative to a larger volume (e.g., amount of waterrelative to air).

“Multi-modal” as used herein refers to the ability of a device tocommunicate using multiple protocols and/or bandwidths. Examples ofmultimodal may include being capable of communication using two to moreof: Bluetooth; Bluetooth Low Energy; WiFi; WiFi RT; GPS; ultrasonic;infrared protocols and/or mediums.

“Node” as used herein means a device including at least a processor, adigital storage and a wireless transceiver.

“Performance” as used herein may include a metric of an action orquantity. Examples of Performance may include metrics of: number ofprocesses completed, energy efficiency; length of service; cost ofoperation; quantity of goods processed or manufacture; quality of goodsprocessed or manufacture; yield; and human resources required.

“Performance Level” as used herein means one or both of a quantity ofactions executed and a quality of actions.

“Property” as used herein shall mean one or more real estate parcelssuitable for a deployed Structure that may be modeled in an AVM.

“Ray” as used herein refers to a straight line including a startingpoint and extending indefinitely in a direction.

“Sensor” as used herein refers to one or more of a solid state,electro-mechanical, and mechanical device capable of transducing aphysical condition or Property into an analogue or digitalrepresentation and/or metric.

“Smart Device” as used herein includes an electronic device including,or in logical communication with, a processor and digital storage andcapable of executing logical commands.

“Structure” as used herein refers to a manmade assembly of partsconnected in an ordered way. Examples of a Structure in this disclosureinclude a building; a sub-assembly of a building; a bridge, a roadway, atrain track, a train trestle, an aqueduct; a tunnel a dam, and aretainer berm.

“Structural Message” as used herein refers to a logical communicationgenerated by automation (such as a Sensor or machine) incorporated into,affixed to or operated within or proximate to a Structure.

“Structural Messaging” as used herein refers to an action that generatesand/or transmits a Structural Message.

“Total Resources” as used herein shall mean an aggregate of one or moretypes of resources expended over a time period.

“Transceive” as used herein refers to an act of transmitting andreceiving data.

“Transceiver” as used herein refers to au electronic device capable ofone or both of transmitting and receiving data.

“Vantage Point” as used herein refers to a specified location which maybe an actual location within a physical Structure or a virtualrepresentation of the actual location within a physical Structure.

“Vector” as used herein refers to a magnitude and a direction as may berepresented and/or modeled by a directed line segment with a length thatrepresents the magnitude and an orientation in space that represents thedirection.

“Virtual Structure” (“VS”): as used herein shall mean a digitalrepresentation of a physical Structure suitable for use. The VS mayinclude Design Features and As Built Features. The VS may be included aspart of an AVM.

According to the present invention, multiple Nodes are deployed in orproximate to a Structure to provide data quantifying positions of theNodes relative to each other and/or aspects of a Structure. In addition,Sensors may be deployed with known positions relative to one or moreNodes, the Sensors are operative to quantify conditions in anenvironment available to the sensor. The data quantifying respectiveconditions registered by the Sensors may referenced to generate a statusand/or condition of one or more of: a deployed Structure, a Structure inthe process of being built; and/or a Structure in the process of beingretrofitted with a position of quantified conditions determined basedupon use of a self-verifying array of Nodes.

In some embodiments, a location of one or more Sensors may be generatedaccording to the methods herein. The location may be in relation to oneor more of: a home position; a position of an Agent; and a position ofone or more Reference Position Transceivers. An Agent may be guided to aSensor and/or an area of interest based upon a Sensor reading usingorienteering methods and apparatus presented herein. For example, acontroller may receive Sensor data quantifying temperature and humiditythat exceed an optimal range of temperature and humidity (e.g., the dataquantifying temperature and humidity may indicate an environmentconducive to termites in the Structure, or simply inefficient insulationfrom an outside environment). Using Orienteering, an Agent may be guidedto one or both of the Sensors that generated the data and an area ofinterest indicated by the measured data. A user interface may includehuman ascertainable indications of the conditions quantified and/or thelocation of the conditions quantified.

Additional examples may include guiding an Agent to a Sensor to replacea power source, such as a battery or battery pack. Other exemplary powersources include an antenna or array of antennas tuned to receive ambientenergy and recharge an energy storage device (such as a battery).

Referring now to FIG. 1A a block diagram illustrates various aspects ofthe present invention and interactions between the respective aspects.The present invention includes an AVM 111 of a Structure that includesAs Built Features as well as the generation and inclusion of As BuiltFeatures, based upon location and direction-specific data capture, isdiscussed more fully below. Data may be transmitted and received via oneor both of digital and analog communications, such as via a wirelesscommunication medium 117.

According to the present invention, one or more Deployment PerformanceMetrics 112 are entered into automated apparatus in logicalcommunication with the AVM 111. The Deployment Performance Metric 112may essentially include a purpose to be achieved during Deployment of amodeled Structure. By way of non-limiting example, a DeploymentPerformance Level may include one or more of: a production or quantity;quality; yield; scalability; a level of energy efficiency; a level ofwater consumption; mean time between failure for equipment included inthe Structure; mean time between failure for machinery installed in theStructure; a threshold period of time between repairs on the Structure;a threshold period of time between upgrades of the Structure; a targetmarket value for a Property; a target lease or rental value for aProperty; a cost of financing for a Property; Total Cost of Ownership ofa Property; Total Cost of Deployment of a Property or other quantifiableaspect.

In some embodiments, Deployment Performance Metrics may be related to afungible item, such as a measurement of energy (e.g., kWh ofelectricity, gallon of fuel oil, cubic foot of gas, etc.); man-hours ofwork; trade medium (e.g., currency, bitcoin, stock, security, optionetc.); parts of manufactured volume of material processed or otherquantity. Relating multiple disparate Deployment Performance Metrics toa fungible item allows disparate Performance Metrics to be compared forrelative value.

Modeled Performance Levels 113 may also be entered into the automatedapparatus in logical communication with the AVM 111. The ModeledPerformance Levels 113 may include an appropriate level of Performanceof an aspect of the Structure in the AVM affected by the DeploymentPerformance Metric 112. For example, a Performance Level 113 for energyefficiency for a Structure modeled may include a threshold ofkilowatt-hours of electricity consumed by the Structure on a monthlybasis. Similarly, a target market value or lease value may be athreshold pecuniary amount. In some embodiments, a pecuniary amount maybe according to a period of time, such as monthly, or a term of years.

Empirical Metrics Data 114 may be generated and entered into theautomated apparatus on an ongoing basis. The Empirical Metrics Data 114will relate to one or more of the Deployment Performance Metrics and maybe used to determine compliance with a Deployment Performance Leveland/or a Performance Levels. Empirical Metrics Data 114 may include, byway of non-limiting example, one or more of: a unit of energy; a unit ofwater; a number of service calls; a cost of maintenance; a cost ofupgrades; equipment details, design details, machinery details,identification of human resources deployed; identification oforganizations deployed; number of human resources; demographics of humanresources (e.g., age, gender, occupations, employment status, economicstatus, requiring assistance with basic living necessities; and thelike); percentage of time Structure is occupied; purpose of occupancy(e.g., primary residence, secondary residence, short-term rental,long-term lease, etc.); Sensor readings (as discussed more fully below);man-hours required for Structure repair, maintenance, or upgrades; andtotal currency (or other fungible pecuniary amount) expended on behalfof a Structure or Property.

In addition to Empirical Metrics Data 114, Lead Actions and expected LagBenefits 115 that may cause an effect on one or both of a DeploymentPerformance Level 112 and a Performance Level 113, may be entered intothe automated apparatus. A Lead Action may include an action expected toraise, maintain or lower an Empirical Metrics Data 114. For example, anaction to install water efficient plumbing fixtures may be scheduled inorder to improve water consumption metrics. Similar actions may relateto electrically efficient devices, or automatic electric switches beinginstalled; preventive maintenance being performed; Structure automationdevices being installed and the like. Other Lead Actions may includelimiting a demographic of occupants of a Structure to a certaindemographic, such as senior citizens. An expected benefit may bemeasured in Lag Benefit measurements, such as those described asEmpirical Metrics Data 114, or less tangible benefits, such as occupantsatisfaction.

The automated apparatus may also be operative to calculate FuturePerformance 116 based upon one or more of: AVM Model with As Built Data111; Deployment Performance Metrics 112; Modeled Performance Levels 113and Empirical Metrics Data 114. Future Performance may be calculated interms of an appropriate unit of measure for the aspect for whichPerformance is calculated, such as, for example: an energy unit; manhours; mean time between failures and dollar or other currency amount.

Calculation of Future Performance 116 may be particularly useful tocalculate Total Resources calculated to be required to support aparticular Structure, group of Structures, properties and/or group ofproperties over a tem of years (“Total Resources Calculated”). TotalResources Calculated may therefore be related to calculations of FuturePerformance 116 and include, for example, one or more of: energy units;water units; man hours; equipment; machinery and dollars (or othercurrency or fungible item). In some embodiments, calculations of FuturePerformance may include a Total Cost of Ownership for a term of years.For example, a Total Cost of Ownership for a Property may include apurchase amount and amounts required for maintenance, repair andupgrades from day one of Deployment through twenty years of Deployment(a shorter or longer term of years may also be calculated).

Accordingly, some embodiments may include a calculation of TotalResources required that includes a purchase price of a Property with aStructure that incorporates a total cost associated with the Propertyover a specified term of years. The total cost will be based upon theAVM with As Built Data 111; Deployment Performance Metrics 112; ModeledPerformance Levels 113 and Empirical Metrics Data 114.

Moreover, Total Resources required may be aggregated across multipleproperties and. Structures. Aggregation of Properties may be organizedinto Property pools to mitigate risk of anomalies in the Calculation ofFuture Performance. Of course, the benefits of Property ownership and/ormanagement may also be pooled and compared to the Total Resourcesrequired. In various embodiments, different aspects of calculated FuturePerformance 116 may be aggregated and allocated to disparate parties.For example, first aggregation may relate to man hours of techniciantime for Structure repair and maintenance and the fulfillment ofobligations related to the aggregation may be allocated to a firstparty. A second aggregation may relate to machinery Performance andobligations allocated to a second party. A third aggregation may relateto equipment Performance and obligations allocated to a third party.Other aggregations may similarly be allocated to various parties. Insome embodiments, financial obligations incorporating one or both ofacquisition cost and ongoing Deployment costs may be allocated andfinanced as a single loan. Other embodiments include a calculated FuturePerformance cost being incorporated into a purchase price.

An important aspect of the present invention includes definition andexecution of Lead Actions based upon one or more of: the AVM Model withAs Built Data 111; Deployment Performance Metrics 112; ModeledPerformance Levels 113; Empirical Metrics Data 114 and Calculations ofFuture Performance 116.

Referring now to FIG. 1B, an AVM is generally associated with a Propertythat includes real estate parcels 140-143. In some embodiments, one ormore of the following are performed on the Property: monitoring; servicecall; an improvement, a repair, maintenance and an upgrade. The Propertyis identified according to an automated determination of a location anda particular position, elevation and direction are further determinedautomatically within the Property. Smart Devices may be used to accessdata records stored in an AVM according to a unique identifier of aphysical location of the real estate parcels 140-143.

As illustrated, a map of real estate parcels 140-143 is shown with icons140A-142A indicating parcels 140-142 that have virtual Structures140A-142A included in a virtual model associated with the parcels. Otherparcels 143 have an indicator 143A indicating that a virtual model is inprocess of completion.

In some methods utilized by the present invention, data in an AVM may beaccessed via increasingly more accurate determinations. A first level ofgeospatial location determinations may be based upon the real estateparcels 140-143 themselves and a second geospatial determination may bemade according to Reference Position Transceivers (discussed more fullybelow) included within the boundaries of the real estate parcels140-143. Still more accurate location position may be calculatedaccording to one or both of a direction determination and anaccelerometer or other location determination technology. Accordingly,it is within the scope of the present invention to access a record of adesign model for a specific wall portion within a Structure based uponidentification of a particular parcel of real estate parcels 140-143 anda location within a Structure situated within the real estate parcels140-143 and height and direction. Likewise, the present inventionprovides for accessing As Built data and the ability to submit As Builtdata for a specific portion of a Structure based upon an accurateposition and direction determination.

For example, in some embodiments, a first level of locationidentification may include a real estate parcel 140-143 identified basedupon a first wireless communication modality, such as a GPScommunication or a sub-GHz wavelength communication. A second level oflocation identification may include a Structure 141A-143A identified viaone or more of GPS; UWB; Wi-Fi; sonic communications; a sub-GHzwavelength communication and Bluetooth communications. A third level oflocation identification may include an Agent position within a Structure(or Property) based upon logical communications via one or more of: UWB;Wi-Fi; sonic communications; and Bluetooth communications. A fourthlevel of location identification may include a determination of adistance from an item to a Smart Device borne by an Agent, the distancedetermination may be based upon transceiving in a SVAN operating in aBluetooth wavelength, a WiFi wavelength or a sub-GHz wavelength.

In some implementations of the present invention, a Property-uniqueidentifier may be assigned by the AVM and adhere to a standard foruniversally unique identifiers (QUID), other unique identifiers may beadopted from, or be based upon, an acknowledged standard or value. Forexample, in some embodiments, a unique identifier may be based uponCartesian Coordinates, such as global positioning system (GPS)coordinates. Other embodiments may identify a Property according to oneor both of: a street address and a tax map number assigned by a countygovernment or other authority.

In some embodiments, an AVM may also be associated with a larger groupof Properties, such as a manufacturing plant, research and development,assembly, a complex, or other defined arrangement.

As illustrated, in some preferred embodiments, an electronic recordcorrelating with a specific Property may be identified and then accessedbased upon coordinates generated by a GPS device, or other electroniclocation device. The GPS device may determine a location and correlatethe determined location with an AVM record listing model data, As Builtdata, improvement data, Performance data, maintenance data,cost-of-operation data, return-on-investment data and the like.

In another aspect, data generated by Sensors deployed in a Structure maybe aggregated and analyzed according to a Property location and/orStructure location associated with the Sensor/Sensor Cluster/SensorGateway. In this manner, an event may be tracked in a larger geographicarea with numerous data points. For example, an event such as the launchof a rocket may cause data to be generated by multiple Sensor/SensorCluster/Sensor Gateways and tracked across a geographic area. Similarly,a natural event, such as an earthquake, hurricane, wildfire and the likemay be tracked with highly accurate Sensor data across tens, hundreds ormany thousands of data points. Still other events may include, forexample, power usage, power generation, water flow in a hydroelectricsystem, water management in a reservoir system, flooding, release oftoxic components into the environment, etc.

Referring now to FIG. 1C, a relational view of an AVM 100 with a VS 102Bis illustrated, as well as a physical Structure 102A. The AVM 100includes a virtual model stored in digital form with a design aspectthat allows for a physical Structure 102A suitable for use to bedesigned and modeled in a virtual environment. The design aspect mayreference Performance data of features to be included in a VS 102B andalso reference variables quantifying an intended use of the VS 102B. TheVirtual Structure 102B and the AVM 100 may reside in a virtual settingvia appropriate automated apparatus 108. The automated apparatus 108will typically include one or more computer servers and automatedprocessors as described more fully below and may be accessible via knownnetworking protocols.

The Physical Structure 102A may include Transceivers 120 or other typeof Node which may incorporate or be co-located with a Sensor ortransmitter(s) or receiver(s) that monitor or otherwise quantify one ormore conditions in a specified area, which may include, for example anarea of ingress and egress 122, such as a doorway, elevator and/orloading dock. Reference point Transceivers 121A may be used as wirelessreferences of a geospatial position. A wireless Node 123 may also linklogical infrastructure within the Physical Structure 102A with a digitalcommunications network.

In correlation with the design aspect, the present invention includes anAs Built Model 101 that generates a Virtual Structure 102B in thecontext of the AVM 100. The As Built Model 101 includes virtual detailsbased upon As Built data captured on or proximate to a physical site ofa related physical Structure 102A. The As Built data may be captured,for example, during construction or modification of a physical Structure102A.

The As Built Model 101 may include detailed data including imagecaptures via one or more Image Capture Devices 107 and physicalmeasurements of features included in the physical Structure 102A. Thephysical measurements may be during a build phase of the physicalStructure; or subsequent to the build phase of the physical Structure.In some embodiments, original As Built measurements may be supplementedwith additional data Structure data associated with repairs orimprovements are made to the physical Structure. Details of recordablebuild aspects are placed as digital data on a recordable medium 104included in the automated apparatus 108.

The digital data included on a recordable medium 104 may thereforeinclude, for example, one or more of: physical measurements capturingExperiential Data; image data (e.g., digital photos captured with a CCDdevice); laser scans; infrared scans and other measurement mediums. Oneor more records on the recordable medium 104 of an As Built Structuremay be incorporated into the AVM 100 thereby maintaining the parallelnature of the AVM 100 with the physical Structure 102A.

In some embodiments. As Built data on a recordable medium 104 may begenerated and/or captured via an Image Capture Device 107.

As the physical Structure is deployed for use, subsequent measurementsthat generate and/or capture Experiential Data may be made andincorporated into the AVM 100. In addition, a user may access and update103 the AVM 100 to ascertain features of the physical Structure 102Athat have been virtually incorporated into the AVM 100. In someexamples, a tablet, handheld network access device (such as, for examplea mobile phone) or other device with automated location service may beused to determine a general location of a physical Structure 102A. Forexample, a smart phone with global positioning system (GPS) capabilitiesmay be used to determine a physical address of a physical Structure,such as 123 Main Street. Stored records containing data relating to 123Main Street may be accessed via the Internet or other distributednetwork.

In addition to the use of GPS to determine a location of a User Device,the present invention provides for a real estate parcel with a physicalStructure 102A that includes more radio frequency (or other mechanism)location identifiers 121A. Location identifiers 121A may include, forexample, radio transmitters at a defined location that may be used toaccurately identify via triangulation, a position of a user device 106,such as a: tablet, smart phone, or virtual reality device. The positionmay be determined via triangulation, single strength, time delaydetermination or other process. In some embodiments, triangulation maydetermine a location of a user device within millimeters of accuracy.

Other location identifiers may include, by way of non-limiting example,RFID chips, visual markings (e.g., a hash tags or barcode), pins, orother accurately placed indicators. Placement of the locationidentifiers may be included in the AVM and referenced as the location ofthe physical user device is determined. As described above, specificlocation identifiers may be referenced in the context of GPS coordinatesor other more general location identifiers.

Based upon the calculated location of the user device 106, details ofthe physical Structure 102A may be incorporated into the VirtualStructure 102B and presented to a user via a graphical user interface(GUI) on the user device 106.

For example, a user may approach a physical Structure and activate anapp on a mobile user device 106. The app may cause the user device 106to activate a GPS circuit included in the user device and determine ageneral location of the user device 106, such as a street addressdesignation. The general location will allow a correct AVM 100 to beaccessed via a distributed network, such as the Internet. Once accessed,the app may additionally search for one or more location identifiers121A of a type and in a location recorded in the AVM. An AVM mayindicate that one or more RFID chips are accessible in a kitchen, aliving room and each bedroom of a Structure. The user may activateappropriate Sensors to read the RFID chips and determine their location.In another aspect, an AVM 100 may indicate that location identifiers121A are placed at two or more corners (or other placement) of aphysical Structure 102A and each of the location identifiers 121A mayinclude a transmitter with a defined location and at a defined height.The user device 106, or other type of controller, may then triangulatewith the location identifiers 121A to calculate a precise location andheight within the physical Structure.

Similarly, a direction may be calculated via a prescribed movement ofthe user device 106 during execution of code that will record a changein position relative to the location identifiers 121A. For example, auser Smart Device, such as a smart phone or user device 106 may bedirected towards a wall or other Structure portion and upon execution ofexecutable code, the Smart Device may be moved in a generally tangentialdirection towards the wall. The change in direction of the user device106 relative to the location identifiers 121A may be used to calculate adirection. Based upon a recorded position within the Physical Structure102A and the calculated direction, a data record may be accessed in theAVM 100 and a specific portion of the AVM 100 and/or the VirtualStructure 102B may be presented on the user device 106. In otherembodiments, a direction may be chosen, or verified via a mechanisminternal to the Smart Device, such as a compass or accelerometer.

In still another aspect of the present invention, in some embodiments,transmissions from one or more location identifiers 121A may becontrolled via one or more of: encryption; encoding; passwordprotection; private/public key synchronization; or other signal accessrestriction. Control of access to location identifiers 121A may beuseful in multiple respects, for example, a location identifier mayadditionally function to provide access to data, a distributed networkand/or the Internet.

The Virtual Structure 102B may include one or both of: historical dataand most current data relating to aspects viewable or proximate to theuser device 106 while the user device is at the calculated location inthe physical Structure 102A. In this way, the parallel virtual world ofthe AVM 100 and the Virtual Structure 102B may present data from thevirtual world that emulates aspects in the physical world and may beuseful to the user accessing the user device 106, while the user deviceis at a particular physical location. As discussed within this document,data presented via the AVM 100 may include one or more of: design data,As Built data, Experiential Data, Performance data relating to machineryand/or features of the AVM 100 or physical Structure; maintenance data,and annotations.

Annotations may include, for example, a user's or designer's noterecorded at a previous time, a service bulletin, maintenance log,operation instructions or a personal note to a subsequent user, such asa virtual “John Smith was here” guest log indicating who had frequentedthe location. Annotations may include one or both of text and imagedata. For example, an annotation may include an image of the locationcaptured at a given time and date. The image may be of a personalnature, e.g., the living room while the Smith's owned the Structure, ora professional nature, e.g., the living room after being painted by XYZContractor on a recorded date. In some embodiments, annotations may beused to indicate completion of a work order. Recordation of completionof a work order may in turn trigger a payment mechanism for paying anentity contracted to complete the work order. In another aspect,annotations may relate to an AVM or a Virtual Structure as a whole, orto a particular aspect that is proximate to a location of the userdevice within the Virtual Structure.

In some embodiments, details of a proposed use of a Structure and parcelmay be input into a design module and used to specify or recommendfeatures to be included in an AVM 100.

According to the present invention, features of a Structure and parcelare generated within a digital design model and then tracked as thefeatures are implemented in a build process and further tracked inPerformance of the Structure as it is placed into use. To the extentavailable, Performance is tracked in the context of variables relatingto use. Variables may include, for example: a use of the Structure, suchas manufacturing and/or processing; a number of resources accessing in aStructure; demographics of the human resources; number of months peryear the Structure is deployed for use; which months of the year aStructure is deployed for use; which hours of the day the Structure isoccupied; and other relevant information.

As Experiential Sensor Readings are generated, they may be memorializedto generate Experiential Data associated with a physical Structure 102A.The Experiential Data is collected and analyzed via structured queriesand may also be analyzed with artificial intelligence processes such asunstructured queries to derive value. In some embodiments, ExperientialData may also be associated with a human and/or an animal interactingwith the Physical Structure 102A. This may be particularly useful forStructures that are processing plants. Whereas former processing plantswere generally designed and built to mitigate against variability in ahuman 118 and between disparate humans 118, the present invention allowsfor human variability to be monitored via Sensors within device 119.Moreover, the Structure may be modified to optimally interrelate withthe values for variables attributable to a human 118 that will inhabitor otherwise interact with the Physical Structure 102A. Human (and/oranimal) may be quantified with Sensors within device 119 installed on orproximate to the Human 118. Alternatively, Sensors 124 located in, orproximate to, a Physical Structure 102A may be used to monitor humanvariability. Biosensors may be used to provide empirical data of humans118 interacting with a Structure may be analyzed using structured orunstructured queries to device relationships between StructurePerformance and human biometrics. Accordingly, Sensors may be used toquantify interaction between a human 118 and an As Built Structure 101according to physiological and behavioral data, social interactions, andenvironmental factors within the Structure, actions undertaken,movements, and almost any quantifiable aspect.

As Built Features and biometrics may be further utilized to controlvarious Structure automation devices. Structure automation devices mayinclude, by way of non-limiting example one or more of: automated locksor other security devices; thermostats, lighting, heating, chemicalprocessing, cutting, molding, laser shaping, 3D printing, assembly,cleaning, packaging, and the like. Accordingly, a Structure withrecorded As Built Design Features and vibration Sensors may trackactivities in a Structure and determine that a first occupant associatedwith a first vibration pattern of walking is in the Structure. Recordedvibration patterns may indicate that person one is walking down ahallway and automatically turn on appropriated lighting and adjust oneor more of: temperature, sound, and security. Security may includelocking doors for which person one is not programmed to access. Forexample, a first pattern of vibration may be used to automaticallyascertain that a person is traversing an area of a Structure for which ahigh level of security is required or an area that is designated forlimited access due to safety concerns. As Built data has been collected.Other Structure automation may be similarly deployed according to AsBuilt data, occupant profiles, biometric data, time of day, or othercombination of available Sensor readings.

Referring now to FIG. 1D, according to the present invention a virtualmodel is generated that correlates with a physical Structure 102A andincludes virtual representations of As Built features and ExperientialData. As discussed more fully herein, the virtual model may include anAVM with As Built data, such as image data and measurements, includedwithin the model. In addition, Sensor data may be collected over timeand incorporated into the AVM. The AVM may include virtualrepresentations of one or more of: Sensors 155; equipment 156-158;controls 161; infrastructure 159, such as HVAC, utilities, such aselectric and water, gas lines, data lines, etc. and Vantage Points 151.

In some implementations, a virtual reality headset may be worn by a userto provide au immersive experience from a Vantage Point 151 such thatthe user will experience a virtual representation of what it would belike to be located at the Vantage Point 151 within the Structure 152 ata specified point in time. The virtual representation may include acombination of Design Features, As Built data and Experiential data. Avirtual representation may therefore include a virtual representation ofimage data via the visual light spectrum, image data via infrared lightspectrum, noise and vibration reenactment. Although some specific typesof exemplary Sensor data have been described, the descriptions are notmeant to be limiting unless specifically claimed as a limitation and itis within the scope of this invention to include a virtualrepresentation based upon other types of captured Sensor data may alsobe included in the AVM virtual reality representation.

Referring now to FIG. 1E, a user 131 is illustrated situated within anAVM 111. The user 131 will be virtually located at a Vantage Point 137and may receive data 136, including, but not limited to one or more of:image data 134, audio data 135 and Ambient Data 136. The user 131 mayalso be provided with controls 133. Controls 133 may include, forexample, zoom, volume, scroll of data fields and selection of datafields. Controls may be operated based upon an item of Equipment 132within a Field of View 138 of the User 131 located at a Vantage Point137 and viewing a selected direction (Z axis). The user is presentedwith Image Data from within the AVM 111 that includes As Built data andvirtual design data.

Additional examples may include Sensor arrays, audio capture arrays andcamera arrays with multiple data collection angles that may be complete360 degree camera arrays or directional arrays, for example, in someexamples, a Sensor array (including image capture Sensors) may includeat least 120 degrees of data capture, additional examples include aSensor array with at least 180 degrees of image capture; and still otherexamples include a Sensor array with at least 270 degrees of imagecapture. In various examples, data capture may include Sensors arrangedto capture image data in directions that are planar, oblique, orperpendicular in relation to one another.

Referring now to FIG. 2 , a functional block illustrates variouscomponents of some implementations of the present invention. Accordingto the present invention, automated apparatus included in the AVM 201are used to generate a model of a Virtual Structure and may alsoincorporate a model and associated real estate parcel. One or morepieces of equipment that will be deployed in the Property may beincluded into the AVM 201. This equipment may include, for example:machinery 211; building support items 212, and utilities support 213.The AVM 201 may model operational levels 204 during deployment of aStructure and associated machinery and equipment included in the AVM201. Machinery 211 may include, for example, manufacturing tools, robotsor other automation, transport tools, chemical processing machine,physical processing machine, assembly machine, heat processing machine,cooling machine, deposition device, etching device, welding apparatus,cutting apparatus, forming tool, drilling tool, shaping tool, transportmachine, Structure automation, air purification or filter systems, noisecontainment device and the like. Utility support equipment may includecabling, dish antennas, Wi-Fi, water softener, water filter, power,chemical supply, gas supply, compressed air supply and the like, as wellas uptime and downtime associated with a Structure utility and uptimeand down time 243 of one or more aspects of the Structure.

The AVM 201 calculates a predicted Performance of the AVM and generatesOperational Levels 204 based upon the Performance 222, wherein“Performance” may include one or more of: total cost of Deployment 214;operational experience 203 which may include one or both of: objectiveempirical measurements and satisfaction of operator's use an As Builtphysical model based upon the AVM; operational expectations 204, totalmaintenance cost 206, and residual value of an As Built Structurefollowing a term-of-years of occupation and use of an As Built Structurebased upon the AVM. Performance 221 may also be associated with aspecific item of machinery 211.

In another aspect, actual Operational Experience 203 may be monitored,quantified and recorded by the AVM 201. Data quantifying the OperationalExperience 203 may be collected, by way of non-limiting example, fromone or more of: Sensors incorporated into an As Built Structure;maintenance records; utility records indicating an amount of energy 202(electricity, gas, heating oil) consumed; water usage; periodicmeasurements of an As Built Structure, such as an infrared scan ofclimate containment, air flow through air handlers, water flow, waterquality and the like; user surveys and maintenance and replacementrecords.

In still another aspect, a warranty 205 covering one or both of partsand labor associated with an As Built Structure may be tracked,including replacement materials 207. The warranty 205 may apply to anactual Structure, or one or more of machinery 211; building support 212item; and utility support item 213.

The AVM 201 may consider a proposed usage of a Deployment of a Structurebased upon values for Deployment variables and specify aspects of one ormore of: machines 211; building support 212; and utility support 213based upon one or both of a proposed usage and values for Deploymentvariables. Proposed usage may include, for example, how many humanresources will occupy a Structure, demographics of the resources thatwill occupy the Structure; percentage of time that the Structure will beoccupied; whether the Structure is a primary residence; whether theStructure is a leased Property and typical duration of leases enteredinto; and environmental conditions experienced by the Structure, such asexposure to ocean salt, winter conditions, desert conditions, highwinds, heavy rain, high humidity, or other weather conditions.

In another aspect, Deployment may relate to biometrics or other dataassociated with specific occupants of a Structure. Accordingly, in someembodiments, Sensors may monitor biologically related variables ofoccupants and/or proposed occupants. The biometric measurements may beused to determine one or both of Lead Actions and Lag Metrics. LeadActions may include one or more of: use of specific building materials,selection of design aspects; Deployment of Structure equipment;Deployment of machinery; terms of a lease; length of a lease; terms of amaintenance contract; and Structure automation controls.

According to the present invention, design aspects and Structurematerials 210 may also be based upon the proposed usage and values forDeployment variables. For example, a thicker exterior wall with higherinsulation value may be based upon a Structure's location in an adverseenvironment. Accordingly, various demographic considerations andproposed usage of a Structure may be used as input in specifying almostany aspect of a Structure.

In still another consideration, a monetary value for one or more of: aTotal Cost of Deployment (“TCD”). Total Maintenance Cost (“TMC”) and adesired return on investment (“ROI”) for a Property may be used as inputfor one or more design aspects included in an AVM System 200. Total Costof Ownership, TCD, TMC, and ROI may be used to determine optimal valuesof variables 202-205, 210-213 specified in an AVM System 200 andincorporated into an As Built Structure, and other improvements to areal estate parcel.

A Total Cost of Deployment 214 may change based upon a time period 215used to assess the Total Cost of Deployment 214. A ROI may include oneor more of: a rental value that may produce a revenue stream, a resalevalue, a cost of operation, real estate taxes based upon Structurespecifications and almost any other factor that relates to one or bothof a cost and value.

Desirable efficiency and Performance may be calculated according to oneor more of: established metrics, measurement protocols, and pastexperience. The AVM 201 and associated technology and software may beused to support a determination of a TCD. In another aspect, a TCD maybe based upon an assembly of multiple individual metrics, procedures toassess metrics, procedures to adjust and optimize metrics and proceduresto apply best results from benchmark operations. In the course ofmanaging Total Cost of Ownership, in some examples, initial steps mayinclude design aspects that model an optimal design based upon TotalCost of Ownership metrics.

In the following examples, various aspects of Total Cost of Deployment214, Total Maintenance Costs, and associated metrics, are considered inthe context of calculating a target Total Cost of Deployment 214.Accordingly, the AVM may be used to attempt to optimize TCD based on oneor more measured variables.

A designed Structure is ultimately built at a site on a real estateparcel. A build process may be specified, which may provide metrics thatmay be used in a process designed by an AVM 201 and also used as aphysical build proceeds. In some examples, time factors associated witha physical build may be important, and in some examples time factorsassociated with a physical build may be estimated, measured, and actedupon as they are generated in a physical build process. Examples of timefactors may include one or more of: a time to develop and approve siteplans; a time to prepare the site and locate community providedutilities or site provided utilities; a time to lay foundations; a timeto build Structure; a time to finish Structure; a time to installinternal utilities and facilities related aspects; a time to install,debug, qualify and release equipment; and times to start production runsand to certify compliance of production are all examples of times thatcan be measured by various techniques and sensing equipment on aStructure's site. Various time factors for a build are valuable and maybecome increasingly valuable as a physical build proceeds since themonetary investment in the project builds before revenue flows andmonetary investments have clearly defined cost of capital aspects thatscale with the time value of money.

Various build steps may include material flows of various types.Material flow aspects may be tracked and controlled for cost andefficiency. Various materials may lower a build materials cost but raisetime factors to complete the build. Logical variations may be calculatedand assessed in an AVM 201 and optimal build steps may be generatedand/or selected based upon a significance placed upon various benefitsand consequences of a given variable value. Physical build measurementsor Sensor data on physical build projects may also be used as input inan assessment of economic trade-offs.

The equipment deployed may incur a majority of a build cost dependingupon user-defined target values. The AVM may model and presentalternatives including one or more of: cost versus efficiency, quality240, time to build, life expectancy, market valuation over time. A costto build may be correlated with cost to deploy and eventual resale. Anoverall model of a Total Cost of Deployment 214 may include any or allsuch aspects and may also include external. In some examples, the natureof equipment trade-offs may be static, and estimations may be made fromprevious results. In some other examples, changes in technology,strategic changes in sourcing, times of acquisition, and the like mayplay into models of Total Cost of Deployment 214.

In some examples, an initial efficiency of design that incurs largecosts at early stages of a project may have a dominant impact on TotalCost of Deployment 214 when time factors are weighted to real costs. Inother examples, the ability of a Structure to be flexible in itsdeployment or build order over time and to be changed in such flexiblemanners, where such changes are efficiently designed may dominate evenif the initial cost aspects may be less efficient due to the need todesign-in flexibility. As a Structure is built, and as it is operatedthe nature of changing customer needs may create dynamic aspects toestimations of Total Cost of Deployment 214. Therefore, in someexamples, estimates on the expected dynamic nature of demands on aStructure may be modeled against the cost aspects of flexibility tomodel expectations of Total Cost of Deployment 214 given a level ofchange.

In some examples, factors that may be less dependent on extrinsicfactors, such as product demand and the like may still be importantmetrics in Total Cost of Deployment 214. Included in the As Builtfactors may be calculations such as HVAC temperature load, in whichpersonnel and seasonal weather implications may be important. AVM modelsmay include a user interface to receive value useful in the AVM models.In addition, electronic monitoring, via Sensors that may determineenergy consumption, includes for example monitoring any of: electricity,fuel oil, natural gas, propane and the like.

Temperatures may be monitored by thermocouples,semiconductor-junction-based devices or other such direct-measurementtechniques. In other examples, temperature and heat flows may beestimated derived from photon-based measurement, such as surveying theStructure with infrared imaging or the like.

Utility load may be monitored on a Structure-wide basis and/or atpoint-of-use monitoring equipment located at hubs or individual piecesof equipment themselves. Flow meters may be inline, or external to,features such as pipes, wires, or conduits. Gas and liquid flows may bemeasured with physical flow measurements or sound-based measurement. Inother examples, electricity may be monitored as direct currentmeasurements or inferred-inductive current measurement.

In some examples, the nature and design of standard usage patterns of aStructure and an associated environment may have relevance to Total Costof Ownership. For example, usage that includes a larger number ofingress and egress will expose an HVAC system to increased load andusage that includes a significant number of waking hours withinhabitants in the building may incur increased usage of one or more of:machinery 211; building support devices 212; and utilities 234.

The nature and measurement aspects of vibration in the Structure mayalso be modeled and designed as the Structure is built. There may benumerous means to measure vibrations from capacitive- andresistive-based measurements to optical-based measurements that measurea subtle change in distance scale as a means of detecting vibration.Vibration may result from a Stricture being located proximate to aroadway, train, subway, airport, tidal flow, or other significant sourceof relatively consistent vibration. Vibration may also be more periodic,such as earthquake activity. In still another aspect, vibration mayresult from human traffic within the Property. The use ofvibration-monitoring Sensors may indicate various activities that takeplace within the Structure and facilitate more accurate modeling of alife expectancy of various aspects of the Structure as well as machineslocated within the Structure.

Noise levels are another type of vibrational measurement which isfocused on transmission through the atmosphere of the Structure. In somecases, noise may emanate from one location after moving through solidStructure from its true source at another location. Thus, measurement ofambient sound with directional microphones or other microphonic sensingtypes may be used to elucidate the nature and location of noiseemanations. In some cases, other study of the noise emanations may leadto establishment of vibrational measurement of different sources ofnoise. Floors, ceilings, doorways, countertops, windows, and otheraspects of a Structure may be monitored in order to quantify andextrapolate noise levels. Noise and vibrational measurement devices maybe global and monitor a region of a Structure, or they may be inherentlyincorporated into or upon individual equipment of the Structure.

In some examples, models of a Structure (including original models andAs Built models) may include routings of pipes, wires, conduits andother features of a Structure and the installed equipment that haveStructure. Together with models of the building Structure and theequipment placed in the building the various routed Structures may bemarried in a detailed AVM 201.

In another aspect, an AVM 201 may include conflicts between the physicalStructures may be detected and avoided in the design stage at farimproved cost aspects. In some examples, a designer may virtuallyascertain a nature of the conflict and alter a design in virtual spaceto optimize operational aspects. Additionally, in some embodiments, anAs Built model may be generated during and after a Structure is builtfor various purposes. In some examples, a technician may inspect aStructure for conformance of the build to the designed model. In otherexamples, as an As Built Structure is altered to deal with neededchanges, changes will be captured and included in the As Built AVM 201.

In another aspect of the present invention, the AVM 201 may be used togenerate a virtual reality model of a Property, including one or moreStructures that may be displayed via user interface that includes animmersion of the user into a virtual setting. Immersion may beaccomplished, for example, via use of a virtual reality headset withvisual input other than a display screen is limited. In someembodiments, a virtual setting may be generated based upon a location ofthe user. For example, GPS coordinates may indicate a Property and auser may wear a headset that immerses the user in a virtual realitysetting. The virtual reality setting may display one or more virtualmodels of Structures that may be potentially constructed on theProperty.

Embodiments may include models generated using, for example, standardmodeling software such as BIM 360™ field which may support the displayof a Structure design in a very complete level of detail. Modeling of aStructure in its location or proposed location, or in multiple proposedlocations, may be useful from a Total Cost of Ownership perspective,especially from an evaluation of the nature of a site layout includingreal estate Property parcel options and the like.

In some examples, a virtual display observed in the field at the site ofan As Built or proposed build may allow for design changes and designevaluations to be viewed in a space before build is completed. Forexample, a Structure may be completed to the extent that walls, floors,and ceilings are in place. A user may utilize a virtual display tounderstand the layout difference for different designs. Designs may beiterated from designs with the least flexibility to more flexible (yetmore complex) designs.

In some examples, the design systems may include various types offeatures such as building Structure, walls, ducts, utilities, pipes,lighting, and electrical equipment. The design systems are augmentedwith As Built Data and Experiential Data.

The design and modeling systems may be utilized to simulate and projectcost spending profiles and budgeting aspects. The modeling systems maytherefore be useful during the course of an audit, particularly whencomparing actual versus projected spending profiles. The comparison ofvarious spend sequencing may be used to optimize financing costs,maintenance, refurbishing and sequencing. The AVM 201 may be useful toprovide early estimates and for cost tracking against projections. Suchtracking may be visualized as displays across a virtual display of thebuilding, facilities and equipment.

As described above, facing a Node (e.g., a Smart Device) towards an areain a Structure and moving the mobile device in a particular pattern maybe used to ascertain a specific area of the Structure for which AVM 201data should be accessed. A combination of one or more of: image,location, orientation, and other Sensors may also be used to identify tothe mobile device specifically which wall segment, building aspect,machinery, or equipment the device is pointed towards. A location ofmobile device, a height, and an angle of view may also be utilized todetermine aspects of the Structure for which a virtual model is beingrequested.

In some embodiments, a user may be presented with various layers ofdata, including, for example, one or more of: structural aspects of theStructure, plumbing, electrical, data runs, material specifications, orother documentation, including, but not limited to: basic identifyinginformation, installation information, service records, safety manuals,process records, and expected service schedule, among many otherpossibilities.

An additional non-limiting example, data aggregation may include Sensorsgenerating data that is associated with an IoT (Internet ofThings)-based identification. Various IoT devices (or Sensors) mayinclude a digital storage, processor, and transmitter for storing andconveying identifying information. Upon request, an IoT device may relayidentifying information of itself to a human via a communicationsdevice, or to the IoT device's neighbors. It may also possibly conveyinformation received from and/or sent to other internet connecteddevices as well.

As per the above listing, functionality may therefore include modeledand tracked Performance of a Structure and equipment contained withinthe Structure, including consumables 233 used and timing of receipt andprocessing of consumables; modeled and actual maintenance 232, includingquality of maintenance performed; equipment Performance includingyields; Consumables 233 tracking may include a frequency of replacementand quantity of replaced consumables; Utilities 234 tracking may includeprojected and actually units of energy consumed.

In one aspect of the present invention, data related to the position andidentity of substantial elements of a Structure first as designed andthen recorded in their actual placement and installation. This mayinclude locations of building features, such as beams, walls, electricaljunctions, plumbing and etc. as the Structure is designed andconstructed. As part of the Structure model, laser scanning may beperformed on site at various disparate times during construction. Aninitial scan may provide general information relating to the location ofthe Structure in relationship to elements on the Property such asroadways, utilizes such as electricity, water, gas, and sewer toidentify non-limiting examples.

Additional events for scanning may occur during the construction processto capture accurate, three-dimensional As Built point-cloud information.Point cloud may include an array of points determined from image captureand/or laser scanning or other data collection technique of As Builtfeatures. In some examples, captured data may be converted into a 3Dmodel, and saved within a cloud-based data platform.

In some examples other methods of capturing spatially accurateinformation may include the use of drones and optical scanningtechniques which may include high-resolution imagery obtained frommultiple viewpoints. Scanning may be performed with light-based methodssuch as a CCD camera. Other methods may include infrared, ultraviolet,acoustic, and magnetic and electric-field mapping techniques may beutilized.

Structure-related information may include physical features generallyassociated with an exterior of a Structure such as geolocation,elevation, surrounding trees and large landscaping features, undergroundutility locations (such as power, water, sewer, sprinkler system, andmany other possible underground utility features), paving, and pool orpatio areas. Structure-related information may also include featuresgenerally related to a Structure such as underground plumbing locations,stud locations, electrical conduit and wiring, vertical plumbing piping,and HVAC systems or other duct work. The acquisition of the data mayallow the model system to accurately locate these interior and exteriorfeatures. Acquisition of As Built data during different points of theconstruction completion allows measurements to be taken prior to aspectsinvolved in a measurement process being concealed by concrete, drywallor other various building materials.

Data is acquired that is descriptive of actual physical features as thefeatures are built and converted into a 3D model which may be referredto as the “As Built” model. The As Built model will include keycomponents of the Structure and be provided with a level of artificialintelligence that fully describes the key component. In someembodiments, the As Built model may be compared to a design model. Insome implementations, intelligent parameters are associated with keycomponents within the 3D model. For example, key components andassociated information may further be associated with intelligentparameters. Intelligent parameters for the key components may includethe manufacturer, model number, features, options, operationalparameters, whether or not an option is installed (and if so, itsfeatures and dimensions), any hardware associated with the key component(and its manufacturer and serial number), an owner's manual, and servicecontract information, as non-limiting examples. Intelligent parametersassociated with a functional key component, such as HVAC Equipment, mayinclude the manufacturer name, model number, capacity, efficiencyrating, serial number, warranty start date, motor size. SEER rating, anowner's manual associated with the equipment, and service contractinformation.

In another aspect, the AVM system can autonomously and/or interactivelyobtain, store, and process data that is provided to it by Sensorslocated in, on or proximate to components of the Structure, as theStructure is built, or when additions are made to the Structure. Thegeneration, modeling, capture, use, and retention of data relating toPerformances in specific equipment or in some cases, aspects relating tothe design of a Structure, may be monitored by the system.

A Structure may be represented by a three-dimensional model, which maybe integrated with information related to the key components andlaser-scanned location information. This information may be madeavailable to the Stricture owner/Structure builder through a computer,an iPad or tablet, or Smart Device. The resulting system may be usefulto support virtual maintenance support.

The three-dimensional model may support enhancement to thetwo-dimensional views that are typical of paper-based drawings. Althoughthree-dimensional renderings are within the scope of informationdelivered in paper format, a three-dimensional electronic model mayrender dynamic views from a three-dimensional perspective. In someexamples, the viewing may be performed with viewing apparatus thatallows for a virtual reality viewing.

In some examples, a viewing apparatus, such as a tablet or a virtualreality headset, may include orienting features that allow a user suchas a Structure owner, Structure builder, inspector, engineer, designeror the like to view aspects of a model based upon a location, adirection, a height and an angle of view. A current view may besupplemented with various other information relating to featurespresented in the view. In some examples, the interface may be accessiblethrough a virtual reality headset, computer, or mobile device (such asan iPad, tablet, or phone), as non-limiting examples. Utilizing a deviceequipped with an accelerometer, such as a virtual reality headset ormobile device, as non-limiting examples, a viewable section of the modelmay be displayed through the viewing medium (whether on a screen, orthrough a viewing lens), where the viewer's perspective changes as theaccelerometer equipped device moves, allowing them to change their viewof the model. The viewer's Vantage Point may also be adjusted, through acertain user input method, or by physical movement of the user, asnon-limiting examples.

The presented view may be supplemented with “hidden information”, whichmay include for example, depictions of features that were scanned beforewalls were installed. This hidden information may include informationabout pipes, conduits, ductwork and the like. Locations of beams,headers, studs and building Structure may be depicted. In some examples,depiction in a view may include a superposition of an engineeringdrawing with a designed location, in other examples images of an actualStructure may be superimposed upon the image based upon As Built scansor other recordation.

In a dynamic sense, display may be used to support viewing ofhypothetical conditions such as rerouted utilities and rebuild walls andother such Structure. In some examples, graphical- or text-based datamay be superimposed over an image and be used to indicatespecifications, Performance aspects, or other information not related tolocation, shape and size of features in the image.

As presented above, an image may allow for a user to “see through walls”as the augmented reality viewing device simulates a section of a modelassociated with a space displayed via the virtual reality viewingdevice. The viewer's perspective may change as an accelerometer in thevirtual reality viewing device moves. A user may also change a view ofthe AVM to include different layers of data available in the AVM. Theviewer's Vantage Point may also be adjusted by moving about a physicalspace that is represented by the model. To achieve this, it may bepossible to incorporate positioning hardware directly into a buildingrepresented by the virtual model. The positioning hardware may interfacewith an augmented reality device for positioning data to accuratelydetermine the viewing device's orientation and location with millimeterprecision. The positioning hardware may include, for example, a radiotransmitter associated with a reference position and height. Altitude isdifferentiated from height unless specifically referenced since therelative height is typically more important.

Accordingly, a user may access the AVM on site and hold up a SmartDevice, such as an iPad or other tablet, and use the Smart Device togenerate a view inside a wall in front of which the Smart Device ispositioned, based upon the AVM and the location, height and direction ofthe Smart Device position.

In some examples, through the use of an augmented reality device, it mayalso be possible to view data, such as user manuals, etc. of associateddevices in the view of a user, simply by looking at them in the viewinginterface. In other examples, there may be interactive means to selectwhat information is presented on the view.

Various electronic-based devices implementing of the present inventionmay also be viewed in a virtual reality environment withoutaccelerometer such as a laptop or personal computer. A viewable sectionof a model may be displayed on a Graphical User Interface (GUI) and theviewer's Vantage Point may be adjusted, through a user input device.

The ability to track machinery and other components of a system andstore the components associated information—such as, for example, usermanuals, product specifications, and part numbers may allow for muchmore efficient use and maintenance of the components included within aStructure. Additionally, the system model may also maintain Structureowner manuals and warranties and eliminate the need for storage andtracking of hard copy manuals.

In a non-limiting example, a user may access information related to amachinery a Smart Device acting as a Node within it in proximity to themachinery and accessing the parallel model in the Virtual Structure.This access may occur such as by clicking on the machinery in theVirtual Structure model or by scanning the Code label attached tomachinery. In some examples, an IoT-accessible machine may have theability to pair with a user's viewing screen and allow the system modelto look up and display various information. Thus, the user may haveaccess to various intelligent parameters associated with that machinerysuch as service records, a manual, service contract information,warranty information, consumables recommended for use such asdetergents, installation-related information, power supply information,and the like.

In some examples, an AVM system may include interfaces of various kindsto components of the system. Sensors and other operationalparameter-detection apparatus may provide routine feedback ofinformation to the model system. Therefore, by processing thedata-stream with various algorithms autonomous characterization ofoperating condition may be made. Therefore, the AVM system may provide auser with alerts when anomalies in system Performance are recognized. Insome examples, standard Structure maintenance requirements may be sensedor tracked based on usage and/or time and either notification or in somecases scheduling of a service call may be made. In some examples, thealert may be sent via text, email, or both. The Structure user may,accordingly, log back into the Virtual Structure to indicate completionof a maintenance task. Additionally, if appropriate, a vendor of suchservice or maintenance may indicate a nature and completion of workperformed.

By detecting operational status, a Virtual Structure may take additionalautonomous steps to support optimal operation of a system. A VirtualStructure may take steps to order and facilitate shipping of anticipatedparts needed for a scheduled maintenance ahead of a scheduled date for amaintenance event (for example, shipping a filter ahead of time so thefilter arrives prior to the date it is scheduled to be changed). Inanother example, a Virtual Structure may recall notes from an OriginalEquipment Manufacturer (OEM) that could be communicated to a userthrough the Virtual Structure. In still further examples, a VirtualStructure may support a user involved in a real estate transaction byquantifying service records and Performance of a real Property.

In still another aspect, the AVM may establish a standard maintenanceand warranty program based on manufacturers' published data and thesystem's ability to advise Structure owners of upcoming needs and/orrequirements. In other examples, the model system may facilitateallowing for Structure builders, rental companies, or maintenancecompanies to consolidate information for volume discounts on parts ormaintenance items. The model system may also facilitate minimizingunnecessary time expenditure for Structure builders hoping to minimizeneedless service calls for warranty issues. This may also allowStructure builders and rental companies attempting to sell a Structureor a rental to demonstrate that care has been taken to maintain aStructure.

Benefits derived from monitoring and tracking maintenance with a VirtualStructure may include positively reassuring and educating lenders and/orlien holders that their investment is being properly cared for. Inaddition, insurance companies may use access to a Virtual Structure toprovide factual support that their risk is properly managed. In someexamples, a data record in a Virtual Structure model system and how anowner has cared for their Structure may be used by insurance companiesor lenders to ensure that good care is being taken. Maintenance recordsdemonstrating defined criteria may allow insurance companies to offer aStructure owner policy discount. Such criteria may include, for example,installation of an alarm system. Additionally, access to a VirtualStructure may allow municipalities and utilities to use the informationfor accurate metering of utility usage without having to manually checka meter. In the aggregate across multiple Structures, peaks in utilitydemand may then be more accurately anticipated.

In some examples, a Virtual Structure may also be used to assist withStructure improvement projects of various types. In some examples, theStructure improvement projects may include support for building largeradditions and modifications, implementing landscaping projects. Smallerprojects may also be assisted, including in a non-limiting example sucha project as hanging a picture, which may be made safer and easier withthe 3D “as-built” point cloud information. Hidden water piping,electrical conduits, wiring, and the like may be located, or virtually“uncovered”, based on the model database.

During construction of a Structure corresponding to a Virtual Structure,discrete features of the As Built Structure may be identified via anidentification device such as an IoT device or a QR code label. The IDdevice may be integrated to the feature or added during the build scope.Performance monitors may also be simultaneously installed to allowmonitoring of Key Performance Indicators (KPIs) for selected features.In an example, an HVAC system may be added to a Structure duringconstruction and a simultaneously a Performance monitor may be added tothe HVAC system. The Performance monitor may be used to monitor variousKPIs for an HVAC system. These KPIs may include outdoor air temperature,discharge air temperature, discharge air volume, electrical current, andthe like. Similar monitoring capabilities may be installed to allmachinery and utilities systems in a Structure. The combination of thesenumerous system monitors may allow for a fuller picture of theefficiency of operations of various systems.

Use of the Virtual Structure, which may include data values contributedfrom communication of data from the various monitoring systems, mayallow owners to receive periodic reports, such as in a non-limitingsense monthly emails which may show their current total energyconsumption as well as a breakdown of what key components arecontributing to the current total energy consumption.

The systems presented herein may be used by owners and Structuremanagers to make decisions that may improve the cost effectiveness ofthe system. An additional service for Owners may allow the Structureowner to tap into energy-saving options as their Structure ages. As anexample, if a more efficient HVAC system comes on the market, which mayinclude perhaps a new technology Node, the user may receive a “SavingsAlert”. Such an alert may provide an estimated energy savings of therecommended modification along with an estimate of the cost of the newsystem. These estimates may be used to generate a report to the owner ofan estimated associated return-on-investment or estimated payback periodshould the Structure owner elect to replace their HVAC system.

In some examples, an AVM of a Virtual Structure may set a thresholdvalue for the required ROI above which they may be interested inreceiving such an alert with that ROI is achieved. This information willbe based on data derived from actual operating conditions and actualhistorical usage as well as current industry information. Predictivemaintenance and energy savings to key systems via Smart Structure TotalCost of Ownership (“TCO”) branded Sensors.

With the ability to collect and utilize relevant Structure informationwith the model system, the aggregation of data and efficiency experiencefrom numerous systems may allow for analysis of optimization schemes forvarious devices, machinery and other Structure components that includesreal installed location experience. Analysis from the aggregated datamay be used to provide feedback to equipment manufacturers, buildingmaterials fabricators and such suppliers.

Referring to FIGS. 3A-3D, an illustration of the collection of data byscanning a Structure during its construction is provided. In FIG. 3A, adepiction of a site for building a Structure is illustrated. Thedepiction may represent an image that may be seen from above the site.Indications of Property boundaries such as corners 301 and Propertyborders 302 are represented and may be determined based on-site scanningwith Property markings from site surveys or may be entered based onglobal coordinates for the Property lines. An excavated location 303 maybe marked out. Roadways, parking and/or loading areas 304 may belocated. Buried utilities such as buried telephone 305, buried electric306, buried water and sewer 307 are located in the model as illustrated.In some examples, such other site service as a buried sprinkler system308 may also be located.

Referring to FIG. 3B, the excavated location 303 may be scanned orimaged to determine the location of foundation elements. In somenon-limiting examples, a foundational footing 321 along with buriedutilities 322 is illustrated. The buried utilities may include utilitiessuch as electric lines, water supply (whether from a utility or a wellon-location), sewer or septic system lines, and telecommunications linessuch as telephone, cable and internet. Other footing elements 323 may belocated at structural requiring locations as they are built. In someexamples, a scanning system may provide the locational orientationrelative to site-orientation markings. In other examples, aerial imagerysuch as may be obtained with a drone may be used to convert features toaccurate location imagery.

Referring to FIG. 3C, a wall 331 of the Structure in the process ofbuild is illustrated. The Structure may be scanned by a scanning element330. In some examples, a laser three-dimensional Scanner may be used.The wall may have supporting features like top plates 333, headers 336,studs 332, as well as internal items such as pipes 334, electricalconduits, and wires 335. There may be numerous other types of featureswithin walls that may be scanned as they occur such as air ducts, datacables, video cables, telephone cables, and the like.

Referring to FIG. 3D, the wall may be completed with Structurecomponents behind wall facing 340 may no longer be visible. Electricaloutlets 341 and door Structures 342 may be scanned by a scanning element330.

Referring to FIG. 3E, a wireless Node may be fixedly attached to aposition in or proximate to a Structure. In some embodiments, attachmentmay be accomplished during construction and/or retrofitting of astructure, in which case the functionality described herein may be madeoperational to track Agents, materials, equipment, and the like during aconstruction phase, and also track a location of materials and equipmentincluded in the structure. Nodes may be installed as Reference PointTransceivers or be attached to items that dynamically change positions,such as, by way of non-limiting example one or more of: Agents, buildingmaterials, structural components, electrical components, plumbingcomponents, equipment, machines and architectural aspects (e.g., acorner, an arch, an extremity, and the like).

In some non-limiting examples of a wireless Node, a Bluetoothcommunications hub compatible with a standard such as, for exampleBLE5.1 (Bluetooth Low Energy 5.1) or Wi-Fi RTT may be fixedly attachedto a structural component, such as a door header 336 as Node 350 actingas a Reference Point Transceiver. In another example, a Node 351 may actas a Reference Point Transceiver and be attached to a wall stud,preferentially one that has electrical conduit 335 running along it. Insome embodiments, the electrical conduit 335 may supply power to theNode 351. Alternatively, a Node 350 may be configured as part of areceptacle box. In some examples, one or more Nodes 350-351 may bebattery powered. One or more Nodes 350-351 may be powered via electricalsupply wiring 353 from a nearby power conduit 335 so that the Node350-351 may be tied into a centrally powered electrical system.Moreover, the Nodes may be adapted to de-power and de-couple from anetwork based on a power supply status or a power drain change.

FIG. 3F illustrates an exemplary Agent 365 supporting a Smart Device 366with wireless communications components enabling RF communications suchas, one or more of: Cellular, Wi-Fi, Bluetooth, Zigbee, and otherwireless capabilities. The Smart Device 366 may also include devicescapable of receiving and/or transmitting with infrared capabilities. TheSmart Device 366 may also include, or be in logical communication with,transducers capable of emitting sound, and in some examples, infrasoundand/or ultrasonic sound, as well as microphones capable of detectingultrasonic sound and/or infrasound. An Agent 365 may become positionedproximate to a door Structure 342 such that the Agent 365 supportedSmart Device 366 may wirelessly communicate with a Node 362 fixedlyattached to the Structure 342. The Node 362 may be in electricalcommunication with one or more of: a set of protruding antennas 360, anantenna array device 361 (which may include a multitude of antennasseparated at distances efficient for communication and/or locationdetermination). A wireless Node with antennas 362 may be locatedproximate to a typical wall outlet Structure. Any of these Nodes 360-362may communicate with the Smart device for location protocols such asRSSI, Time of Flight, and Angle of Arrival as non-limiting examples. TheNodes 360-362 may have a carefully measured distance characterizationfor each of the antennas that they employ and one of the antennasinvolved in wireless communication may be further characterized as beinga local or global origin point (0,0,0 in Cartesian notation). In otherexamples, none of the antenna locations may be located at a local orglobal origin point, but rather a known offset from a specified originpoint 370 may be characterized for each of the hub antenna locations.

The Agent 365 may proceed through a threshold of the door Structure 342and be located on the other side. Nodes 360-362 may each protrude fromboth sides of a wall and/or may have a second set of antennas located ona distal side of the wall. In other examples, materials used in wallconstruction may be configured to provide minimal interference withwireless signals travelling through the wall materials. Forconfigurations with a second set of antennas, as the user passes throughthe door, a communication between the Smart Device 366 and the Node360-362 may transfer from antennas protruding on a proximate wall sideto antennas protruding on a distal wall side.

A geographic position of a Structure may be calculated via wirelesscommunications, such as those using sub-GHz wavelengths, GPS, or otherlonger-range wavelength a Smart Device from within the Structure. Thegeographic position may be used to indicate a Structure identification.A position within the Structure may be determined based upon one or moreof: an angle of arrival and angle of departure of a wireless signal andone or more timing signals used to determine a distance of the SmartDevice from: a) a Node acting as Reference Point Transceiver; or b) adynamic position Node.

In some embodiments, an angle of departure or an angle of arrival arenot necessary, and a position may be determined by measuring a distanceto three or more positioning reference devices. However, in someembodiments, it may still be useful to compute an angle between thepositioning reference devices and/or the Node.

Additional aspects that may be referenced to determine a location of aNode or Smart Device accurately may include one or more of: relativesignal strength received from wireless transmissions emanating fromanother Nodes; time of arrival of radio signals of wirelesstransmissions emanating from another Node; generating a distance toanother Node based upon a time difference of arrival of radio signals ofwireless transmissions emanating from another Node; or an angle ofarrival and/or angle of departure of a wireless transmission fromanother Node.

The above steps may be repeated for multiple Nodes of various types,including both reference point transceiver Nodes and dynamic positionNodes.

As mentioned above, in some embodiments, wireless communications mayinclude a quantification of a condition within or proximate to aStructure. The condition may include, for example, one or more of: avibration measured with an accelerometer; a temperature of at least aportion of the Structure; an electrical current measurement to equipmentinstalled in the Structure, a number of cycles of operation of equipmentinstalled in the Structure; a number of cycles of operation of anmachinery installed in the Structure; an electrical current measurementto an electrical device located within the Structure; a vibration orother sensor measurement associated with movement of an Agent or personwithin the Structure; or presence of water and/or humidity within theStructure.

A vibration pattern may be associated with a specific occupant andtracking the movement of the specific occupant through the Structure maybe based upon measured vibration patterns. Similarly, a vibrationpattern may be associated with a particular activity of a specificoccupant and the activity of the specific occupant may be tracked withinthe Structure based upon measured vibration patterns.

Referring now to FIG. 4 , according to the present invention, an Agent400 may support a Node with one or more Transceivers. The Transceiversmay include one or more of: a Multi-modality Transceiver 401;Transceivers having a same modality 402; Transceivers of differentmodalities 403: transmitters of a single modality 404; transmitters ofmultiple modalities 405: receivers of a single modality 406; andreceivers of multiple modalities 407. Similarly, a Node deployed as aReference Point Transceiver may include multiple Transceivers,transmitters, and receivers 401-408. The multiple Transceivers,transmitters, and receivers 401-408 may include one or both of:transmitters and receivers of a same modality; and transmitters andreceivers of different modalities.

A modality, as used in conjunction with a Transceiver, transmitter,and/or receiver refers to one or both of a bandwidth of wirelesscommunication and a protocol associated with a bandwidth. By way ofnon-limiting example, a modality, as used in relation to a Transceiver,transmitter, and/or receiver may include: Wi-Fi; Wi-Fi RTT; Bluetooth;UWB; Ultrasonic; sonic; infrared; or other logical communication medium.

FIG. 5 illustrates Nodes with Reference Point Transceivers 501-504 thatmay be deployed in a defined area 506, such as a Structure to determinea location 507 of an Agent 500 supporting a Node 505. Nodes withReference Point Transceivers 501-504 may be fixed in a location andwirelessly communicate in a manner suitable for determination a positionof the Node Transceiver 505 supported by the Agent 500. Transceiving maybe via wireless transmission using one or more bandwidths andcommunication protocols by a Node Transceiver 505 supported by the Agent500.

By way of non-limiting example, Node Transceivers 505 supported by theAgent 500 may be included in, or be in logical communication with, aSmart Device, such as a smart phone, tablet or other Agent-supportabledevice, such as a headgear, ring, watch, wand, pointer with NodeTransceivers 505 able to Transceive with the Reference PointTransceivers 501-504. The Reference Point Transceivers 501-504 mayinclude devices, such as, for example, a radio transmitter, radioreceiver, a light generator, or an image recognizable device. A radiotransmitter may include a router or other Wi-Fi, Bluetooth or othercommunication device for entering into logical communication with acontroller. In some embodiments, Reference Point Transceivers 501-504may include a Wi-Fi router that additionally provides access to adistributed network, such as the Internet. Cartesian Coordinates, PolarCoordinates, Vector values, a GPS position, or other data that may beutilized for one or more of: locating one or both of an Agent 500;indicating a direction of interest; and identifying a Structure ordefined area 506.

A precise location may be determined based upon wireless transmissionsbetween Nodes. Timing determinations—as well as angle of arrival, angleof departure, transmission strength, transmission noise, andtransmission interruptions—may be considered in generating relativepositions of Nodes. Additional considerations may include AI andunstructured queries of transmissions between Nodes and triangulationlogic based upon a measured distance from three or more Reference PointNodes 501-504. For example, a radio transmission or light emission maybe measured, and timing associated with the radio transmission or lightto determine a distance between Nodes. Distances from three referenceposition identifiers 501-503 may be used to generate a position of aNode in consideration. Other methodologies include determination of adistance from one or more Nodes and a respective angle of arrival and/orangle of departure of a radio or light transmission between the Node inconsideration and another Node (Reference Point Node or dynamic positionNode).

Other embodiments may include a device recognizable via image analysisand a camera or other Image Capture Device, such as a CCD device, maycapture an image of three or more Reference Point Nodes 501-504. Imageanalysis may recognize the identification of each of three or more ofthe Reference Point Transceivers 501-504 and a size ratio of therespective image captured Reference Point Transceivers 501-504 may beutilized to calculate a precise position. Similarly, a heightdesignation may be made via triangulation using the position identifiersas reference to a known height or a reference height.

Triangulation essentially includes determining an intersection of threedistances 508-510, each distance 508-510 calculated from a referencepoint 501-504 to an Agent-supported device 505. The present inventionallows for a first distance 508 to be determined based upon a wirelesscommunication in a first modality; and a second distance 509 and a thirddistance 510 determined based upon a wireless communication in a same ordifferent modality as the first modality. For example, a first distance508 may be determined based upon a wireless communication using Wi-Fi; asecond distance 509 may be determined based upon a wirelesscommunication using Bluetooth; and a third communication may bedetermined based upon a wireless communication using ultrasoniccommunication (other combinations of same and/or different communicationmodalities are also within the scope of the present invention).

Referring now to FIG. 6 , an automated controller is illustrated thatmay be used to implement various aspects of the present invention invarious embodiments, and for various aspects of the present invention.Controller 600 may be included in one or more of: a wireless tablet orhandheld smart device, a server, an integrated circuit incorporated intoa Node, appliance, equipment item, machinery or other automation. Thecontroller 600 includes a processor unit 602, such as one or moresemiconductor-based processors, coupled to a communication device 601configured to communicate via a communication network (not shown in FIG.6 ). The communication device 601 may be used to communicate, forexample, with one or more online devices, such as a smart device, aNode, personal computer, laptop, or a handheld device.

The processor 602 is also in communication with a storage device 603.The storage device 603 may comprise any appropriate information storagedevice, including combinations of digital storage devices (e.g., anSSD), optical storage devices, and/or semiconductor memory devices suchas Random Access Memory (RAM) devices and Read Only Memory (ROM)devices.

The storage device 603 can store a software program 604 with executablelogic for controlling the processor 602. The processor 602 performsinstructions of the software program 604, and thereby operates inaccordance with the present invention. The processor 602 may also causethe communication device 601 to transmit information, including, in someinstances, timing transmissions, digital data and control commands tooperate apparatus to implement the processes described above. Thestorage device 603 can additionally store related data in a database 605and database 606, as needed.

Referring now to FIG. 6A, an illustration of an exemplary wireless Node610 configured with a transceiver 624 to wirelessly communicate via oneor more wireless communication Modalities, including a bandwidth andprotocol, such as the Bluetooth 5.1; BLE5.1; Wi-Fi RT and/or GPSstandard is illustrated. As discussed, many different Modalities ofwireless technology may be utilized with the content presented herein,but a BLE5.1 “radio” module is an interesting example since itsstandards provide for angle of arrival (AoA) capability as well as angleof departure (AoD) and a distance determination based upon a timingsignal. With AoA/AoD a designed antenna array 625 can be used by an RFTransceiver 624 to measure a phase shift amongst multiple antennaelements to estimate distance differences between the antennas and toextract an angle from the antenna array to the source of radiation. ABLE5.1-consistent multichip transceiver 624 may include circuitry andsoftware code to perform the acquisition of data and determine the angleof arrival in some examples. In other examples, a BLE5.1-consistentmultichip transceiver 624 may control the acquisition of data from anantenna array while streaming the data to off module processingcapabilities. The BLE5.1-consistent Node 610 may contain functionalblocks of circuitry for peripheral 620 control. The peripherals mayinclude a connection to external host controllers/MCUs 621. Theperipheral 620 control may also interact with peripheral and IoT Sensorsand other devices 622.

The BLE5.1-consistent Node 610 may include a processing element 623which may have its own memory of different types as well as capabilitiesfor encryption of data. The BLE5.1 consistent Node 610 may also haveTransceiver 624. This circuitry may include Baseband and RF functions aswell as control the AoA functions and the self-verifying arrayfunctions. The Bluetooth communications 624 may receive signals throughan on-module antenna 625 or an external antenna or array of antennas mayprovide external RF input 626. The BLE5.1-consistent Node 610 mayinclude functional circuitry blocks for control of Security functions627. cryptogenerations, random number generation and the like. TheBLE5.1-consistent Node 610 may include functional blocks for powermanagement 628.

The BLE5.1-consistent Node 610 may be operative for quantification oftemperature aspects of the Node 610, battery-control functions andpower-conversion functions. An external power source 633 may be includedto provide electrical energy to a power management unit 628 which, insome examples, may be from a battery unit, or a grid connected powersupply source in other examples. The BLE5.1-consistent Node 610 mayinclude functions for control of timing and triggering 629. In a relatedsense, the BLE5.1-consistent Node 610 may include functions for clockmanagement 630 within the module. The BLE5.1-consistent Node 610 mayalso include circuit elements that are always-on 631 to allow externalconnections 632 to interact with the device and perhaps awake it from adormant state. There may also be other customized and/or genericfunctions that are included in a BLE5.1-consistent Node 610 and/ormultichip module.

Referring now to FIG. 6B, a Node 610 included in a higher orderdeployment assembly is illustrated. A deployment Node 650 may be inlogical communication with one or more of: sensors, customized controlcommands, antenna array designs and the like.

A Node 650 may include multiple antennas or antenna arrays 651-656. Asdescribed previously, the Node 650 may include a transceiver module 610,and in some examples, the transceiver module may includeBluetooth-adherent aspects. Communications received via an antenna651-656 may be directly ported into the transceiver module 610.Embodiments may also include routing particular antenna/antenna arrayoutputs to the transceiver module 610 in a controlled and timedsequence. A processing Module 670 may coordinate a connection of theNode 650 to external peripherals.

In some examples, circuitry 680 to logically communicate with one ormore of: a Peripheral, a data Connection, Cameras and Sensorscontrollers, and components to perform data and image acquisition ofvarious kinds, or it may interface external components with the Node650.

The Node 650 may also include its own power management unit 660 whichmay take connected power or battery power or both and use it to provethe various power needs of the components of the assembly. The Node 650may have its own processing modules 670 or collections of differenttypes of processing functions which may have dedicated memory components671. In some examples, specialized processing chips of various kindssuch as Graphical Processing Units and fast mathematics functioncalculators as well as dedicated artificial intelligence processingchips may be included to allow the Node 650 to perform variouscomputational functions including location determination of wirelesslyconnected devices amongst other functions. There may be numerous otherfunctions to include in a Node 650 and alternative types of devices toperform the functions presented herein.

In some examples as illustrated in FIG. 6C antenna arrays 690, 691 maybe assembled into a “Puck” shown as Node 650 wherein the antenna arraysare configured with antenna designs which have directional aspects tothem. Directional aspects may mean that the antennas may be sensitive toincident radiation coming from a certain direction but not sensitive toradiation coming from a different direction. Antenna arrays 690, 691 mayinclude antennas that may have maximized signals for a particularincident waveform, the identification of which antenna may provide orsupplement angle of incidence calculations.

A directional antenna may include, for example, an antenna with RFshielding over some portion of an antenna's circumference. For example,270° (or some other subset of a 360° circumference of an antenna), or anantenna array may have RF shielding to block and/or reflect back an RFsignal towards the antenna-receiving portion. Other directional antennasmay include a shield blocking less than 360° of RF transmissions thatrotates around a receiving portion of an antenna and only receives RFcommunications from a direction of an opening in the shield. Shieldedantennas may provide improved determination of a direction from which awireless transmission is being received from, since RF noise is blockedfrom a significant portion of a reception sphere.

Referring now to FIG. 7 , a block diagram of an exemplary mobile device702 is illustrated. The mobile device 702 comprises an optical capturedevice 708 to capture an image and convert it to machine-compatibledata, and an optical path 706, typically a lens, an aperture or an imageconduit to convey the image from the rendered document to the opticalcapture device 708. The optical capture device 708 may incorporate aCCD, a Complementary Metal Oxide Semiconductor (CMOS) imaging device, oran optical Sensor 724 of another type.

A microphone 710 and associated circuitry may convert the sound of theenvironment, including spoken words, into machine-compatible signals.Input facilities may exist in the form of buttons, scroll wheels, orother tactile Sensors such as touchpads. In some embodiments, inputfacilities may include a touchscreen display.

Visual feedback to the user is possible through a visual display,touchscreen display, or indicator lights. Audible feedback 734 may comefrom a loudspeaker or other audio transducer. Tactile feedback may comefrom a vibrate module 736.

A motion Sensor 738 and associated circuitry convert the motion of themobile device 702 into machine-compatible signals. The motion Sensor 738may comprise an accelerometer that may be used to sense measurablephysical acceleration, orientation, vibration, and other movements. Insome embodiments, motion Sensor 738 may include a gyroscope or otherdevice to sense different motions.

A location Sensor 740 and associated circuitry may be used to determinethe location of the device. The location Sensor 740 may detect GlobalPosition System (GPS) radio signals from satellites or may also useassisted GPS where the mobile device may use a cellular network todecrease the time necessary to determine location. In some embodiments,the location Sensor 740 may use radio waves to determine the distancefrom known radio sources such as cellular towers to determine thelocation of the mobile device 702. In some embodiments these radiosignals may be used in addition to GPS.

The mobile device 702 comprises logic 726 to interact with the variousother components, possibly processing the received signals intodifferent formats and/or interpretations. Logic 726 may be operable toread and write data and program instructions stored in associatedstorage or memory 730 such as RAM, ROM, flash, or other suitable memory.It may read a time signal from the clock unit 728. In some embodiments,the mobile device 702 may have an on-board power supply 732. In otherembodiments, the mobile device 702 may be powered from a tetheredconnection to another device, such as a Universal Serial Bus (USB)connection.

The mobile device 702 also includes a network interface 716 tocommunicate data to a network and/or an associated computing device.Network interface 716 may provide two-way data communication. Forexample, network interface 716 may operate according to the inter netprotocol. As another example, network interface 716 may be a local areanetwork (LAN) card allowing a data communication connection to acompatible LAN. As another example, network interface 716 may be acellular antenna and associated circuitry which may allow the mobiledevice to communicate over standard wireless data communicationnetworks. In some implementations, network interface 716 may include aUniversal Serial Bus (USB) to supply power or transmit data. In someembodiments other wireless links may also be implemented.

As an example of one use of mobile device 702, a reader may scan somecoded information from a location marker in a Structure with the mobiledevice 702. The coded information may be included on apparatus such as ahash code, bar code, RFID or other data storage device. In someembodiments, the scan may include a bit-mapped image via the opticalcapture device 708. Logic 726 causes the bit-mapped image to be storedin memory 730 with an associated timestamp read from the clock unit 728.Logic 726 may also perform optical character recognition (OCR) or otherpost-scan processing on the bit-mapped image to convert it to text.Logic 726 may optionally extract a signature from the image, for exampleby performing a convolution-like process to locate repeating occurrencesof characters, symbols or objects, and determine the distance or numberof other characters, symbols, or objects between these repeatedelements. The reader may then upload the bit-mapped image (or text orother signature, if post-scan processing has been performed by logic726) to an associated computer via network interface 716.

As an example of another use of mobile device 702, a reader may capturesome text from an article as an audio file by using microphone 710 as anacoustic capture port. Logic 726 causes audio file to be stored inmemory 730. Logic 726 may also perform voice recognition or otherpost-scan processing on the audio file to convert it to text. As above,the reader may then upload the audio file (or text produced by post-scanprocessing performed by logic 726) to an associated computer via networkinterface 716.

A directional Sensor 741 may also be incorporated into the mobile device702. The directional Sensor may be a compass and produce data based upona magnetic reading or based upon network settings.

Referring now to FIG. 8 , additional apparatus and methods fordetermining a geospatial location and determination of a direction ofinterest may include one or both of an enhanced Smart Device and a SmartDevice in logical communication with wireless position devices 803-810.The importance of geospatial location and determination of a directionof interest is discussed in considerable detail above. As illustrated, aSmart Device 801 may be in logical communication with one or morewireless position devices 803-810 strategically located in relation tothe physical dimensions of the Smart Device. For example, the SmartDevice 801 may include a smart phone or tablet device with a userinterface surface 820 that is generally planar. The user interfacesurface 820 will include a forward edge 818 and a trailing edge 819.

In some preferred embodiments, the Smart Device will be fixedly attachedto a smart receptacle 802. The smart receptacle 802 may include anappearance of a passive case, such as the type typically used to protectthe Smart Device 801 from a damaging impact. However, according to thepresent invention, the smart receptacle 802 will include digital and/oranalog logical components, such as wireless position devices 803-810.The wireless position devices 803-810 include circuitry capable ofreceiving wireless transmissions from multiple wireless positionalreference Transceivers 811-814. The wireless transmissions will includeone or both of analog and digital data suitable for calculating adistance from each respective reference point 811-814.

In some embodiments, the smart receptacle 802 will include a connector815 for creating an electrical path for carrying one or both ofelectrical power and logic signals between the Smart Device 801 and thesmart receptacle 802. For example, the connector 815 may include amini-USB connector or a lightening connector. Additional embodiments mayinclude an inductive coil arrangement for transferring power.

Embodiments may also include wireless transmitters and receivers toprovide logical communication between the wireless position devices803-810 and the Smart Device 801. Logical communication may beaccomplished, for example, via one or more of: Bluetooth, ANT, andinfrared media.

Reference Transceivers 811-814 provide wireless transmissions of datathat may be received by wireless position devices 803-810. The wirelesstransmissions are utilized to generate a position of the respectivewireless position devices 803-810 in relation to the referenceTransceivers 811-814 providing the wireless transmissions to thewireless position devices 803-810. The wireless position devices 803-810are associated with one or more of: a position in a virtual model; ageographic position; a geospatial position in a defined area, such asStructure; and a geospatial position within a defined area (such as, forexample a Property).

According to the present invention, a Smart Device may be placed into acase, such as a smart receptacle 802 that includes two or more wirelessposition devices 803-810. The wireless position devices 803-810 mayinclude, for example, one or both of: a receiver and a transmitter, inlogical communication with an antenna configured to communicate withreference Transceivers 811-814. Communications relevant to locationdetermination may include, for example, one or more of: timing signals;SIM information; received signal strength; GPS data; raw radiomeasurements; Cell-ID; round trip time of a signal; phase; and angle ofreceived/transmitted signal; time of arrival of a signal; a timedifference of arrival; and other data useful in determining a location.

The Nodes 803-810 may be located strategically in the case 802 toprovide intuitive direction to a user holding the case 802, and also toprovide a most accurate determination of direction. Accordingly, aforward Node 803 may be placed at a top of a Smart Device case and arearward Node 804 may be placed at a bottom of a Smart Device case 802.Some embodiments each of four corners of a case may include a Node 805,806, 807, 808. Still other embodiments may include a Node 809 and 810 oneach lateral side.

The present invention provides for determination of a location of two ormore wireless positioning devices 803-810 and generation of one or moredirectional Vectors 817 and/or Rays based upon the relative position ofthe wireless positioning devices 803-810. For the sake of convenience inthis specification, discussion of a Vector that does not includespecific limitations as to a length of the Vector and is primarilyconcerned with a direction, a Ray of unlimited length may also beutilized. In some embodiments, multiple directional Vectors 817 aregenerated and a direction of one or more edges, such as a forward edge,is determined based upon the multiple directional Vectors 817.

According to the present invention, a geospatial location relative toone or more known reference points is generated. The geospatial locationin space may be referred to as having an X,Y position indicating aplanar designation (e.g., a position on a flat floor), and a Z position(e.g., a level within a Structure, such as a second floor) may begenerated based upon indicators of distance from reference points.Indicators of distance may include a comparison of timing signalsreceived from wireless references. A geospatial location may begenerated relative to the reference points. In some embodiments, ageospatial location with reference to a larger geographic area isassociated with the reference points, however, in many embodiments, thecontroller will generate a geospatial location relative to the referencepoint(s) and it is not relevant where the position is located inrelation to a greater geospatial area.

In some embodiments, a position of a Smart Device may be ascertained viaone or more of: triangulation; trilateration; and multilateration (MLT)techniques.

A geospatial location based upon triangulation may be generated basedupon a controller receiving a measurement of angles between the positionand known points at either end of a fixed baseline. A point of ageospatial location may be determined based upon generation of atriangle with one known side and two known angles.

A geospatial location based upon trilateration may be generated basedupon a controller receiving wireless indicators of distance and geometryof geometric shapes, such as circles, spheres, triangles and the like.

A geospatial location based upon multilateration may be generated basedon a controller receiving a measurement of a difference in distance totwo reference positions, each reference position being associated with aknown location. Wireless signals may be available at one or more ofperiodically, within determined timespans, and continually. Thedetermination of the difference in distance between two referencepositions provides multiple potential locations at the determineddistance. A controller may be used to generate a plot of potentiallocations. In some embodiments, the potential determinations generallyform a curve. Specific embodiments will generate a hyperbolic curve.

The controller may be programmed to execute code to locate an exactposition along a generated curve, which is used to generate a geospatiallocation. The multilateration thereby receives as input multiplemeasurements of distance to reference points, wherein a secondmeasurement taken to a second set of stations (which may include onestation of a first set of stations) is used to generate a second curve.A point of intersection of the first curve and the second curve is usedto indicate a specific location.

In combination with, or in place of directional movement of a SmartDevice 801 in order to quantify a direction of interest to a user, someembodiments may include an electronic and/or magnetic directionalindicator that may be aligned by a user in a direction of interest.Alignment may include, for example, pointing a specified side of adevice, or pointing an arrow or other symbol displayed upon a userinterface on the device towards a direction of interest.

In a similar fashion, triangulation may be utilized to determine arelative elevation of the Smart Device as compared to a referenceelevation of the reference points.

It should be noted that although a Smart Device is generally operated bya human user, some embodiments of the present invention include acontroller, accelerometer, and data storage medium, Image CaptureDevice, such as a Charge Coupled Device (“CCD”) capture device and/or aninfrared capture device being available in a handheld or =maimed vehicleor other Agent.

An unmanned vehicle may include for example, an unmanned aerial vehicle(“UAV”) or an unmanned ground vehicle (“UGV”), such as a unit withwheels or tracks for mobility. A radio control unit may be used totransmit control signals to a UAV and/or a UGV. A radio control unit mayalso receive wireless communications from the unmanned vehicle.

In some embodiments, multiple unmanned vehicles may capture data in asynchronized fashion to add depth to the image capture and/or a3-dimensional and 4-dimensional (over time) aspect to the captured data.In some implementations, a UAV position will be contained within aperimeter and the perimeter will have multiple reference points to helpeach UAV (or other unmanned vehicle) determine a position in relation tostatic features of a building within which it is operating and also inrelation to other unmanned vehicles. Still other aspects includeunmanned vehicles that may not only capture data but also function toperform a task, such as paint a wall, drill a hole, cut along a definedpath, or other function. As stated throughout this disclosure, thecaptured data may be incorporated into an AVM.

In still other embodiments, captured data may be compared to a libraryof stored data using recognition software to ascertain and/or affirm aspecific location, elevation, and direction of an image capture locationand proper alignment with the virtual model. Still other aspects mayinclude the use of a compass incorporated into a Smart Device.

By way of non-limiting example, functions of the methods and apparatuspresented herein may include one or more of the following factors thatmay be modeled and/or tracked over a defined period of time, such as,for example, an expected life of a build (such as 10 years or 20 years).

Referring now to FIG. 8A, in some embodiments, Nodes 803A-810A may beincorporated into a Smart Device 801A and not require a smart receptacleto house Nodes 803-810. Nodes 803A-810A that are incorporated into aSmart Device, such as a smart phone or smart tablet, will includeinternal power and logic connections and therefore not require wirelesscommunication between the controller in the Smart Device 801A and theNodes 803A-810A.

A Smart Device 801A with integrated Nodes 803-810 and a Smart Device 801with Nodes 803-810 in a smart receptacle 802 may provide a directionalindication, such as a directional Vector 817 817A, without needing tomove the Smart Device from a first position to a second position since adirectional Vector may be determined from a relative position of a firstNodes 803-810 and a second wireless positional device Nodes 803-810.

In exemplary embodiments, as described herein, the distances may betriangulated based on measurements of Wi-Fi strength at two points.Wi-Fi signal propagates outward as a wave, ideally according to aninverse square law. Ultimately, a feature of the present inventionrelies on measuring relative distances at two points. In light of thespeed of Wi-Fi waves and the real-time computations involved inorienteering; these computations need to be as computationally simple aspossible. Thus, depending upon a specific application and mechanism forquantifying a condition, such as a measurement, various coordinatesystems may be desirable. In particular, if the Smart Device moves onlyin a planar direction while the elevation is constant, or only at anangle relative to the ground, the computation is more simple.

One exemplary coordinate system includes a polar coordinate system. Oneexample of a three-dimensional polar coordinate system is a sphericalcoordinate system. A spherical coordinate system typically comprisesthree coordinates: a radial coordinate, a polar angle, and an azimuthalangle (r, θ, and φ, respectively, though θ and φ are occasionallyswapped conventionally).

By way of non-limiting example, suppose Point 1 is considered the originfor a spherical coordinate system (i.e., the point (0, 0, 0)). EachWi-Fi emitter e1, e2, e3 can be described as points (r1, θ1, φ1), (r2,θ2, φ2), and (r3, θ3, φ3), respectively. Each of the r_(i)'s (1≤i≤3)represent the distance between the Wi-Fi emitter and the Wi-Fi receiveron the Smart Device.

It is understood that in some embodiments, an azimuth may include anangle, such as a horizontal angle determined in an arcuate manner from areference plane or other base direction line, such as an angle formedbetween a reference point or reference direction; and line (Ray orVector) such as a Ray or Vector generated from or continuing to; a SmartDevice, or a positional Sensor in logical communication with a SmartDevice or other controller. In preferred embodiments the Ray or Vectormay be generally directed from a reference point Transceiver towards,and/or intersect one or more of: an item of interest; a point ofinterest; an architectural aspect (such as a wall, beam, header, corner,arch, doorway, window, etc.); an installed component that may act as areference in an AVM (such as for example, an electrical outlet, a lightfixture, a plumbing fixture, an architectural aspect; an item ofequipment; an appliance; a multimedia device, etc.); another referencepoint Transceiver or other identifiable destination. Embodiments includea position of the Transceiver being determined via use of a polarcoordinate system. The polar coordinate system may include a sphericalcoordinate system or a cylindrical coordinate system.

Accordingly, in some embodiments, spherical coordinate system mayinclude reference point Transceiver that is capable of determining anangle of departure of a location signal and a Transceiver that iscapable of determining an angle of arrival of the location signal; oneor both of which may be used to facilitate determination of anapplicable azimuth.

According to various embodiments of the present invention, one or bothof an angle of departure and an angle of arrival may therefore beregistered by a Transceiver that is transmitting and/or receivingwireless signals (e.g., radio frequency, Bluetooth 5.1, sonic frequency,or light frequency).

In some embodiments, orienteering occurs in a Structure, in whichTransceivers, (including, for example, one or more of: Wi-FiTransceivers, UWB Transceivers, Bluetooth Transceivers, infraredTransceivers and ultrasonic Transceivers) may be located above and/orbelow an Agent. In these embodiments, a cylindrical coordinate systemmay be more appropriate. A cylindrical coordinate system typicallycomprises three coordinates: a radial coordinate, an angular coordinate,and an elevation (r, θ, and z, respectively). A cylindrical coordinatesystem may be desirable where, for example, all Wi-Fi emitters have thesame elevation.

Referring now to FIG. 8B, in some embodiments, one or both of a SmartDevice 801 and a smart receptacle 802 may be rotated in a manner (suchas, for example in a clockwise or counterclockwise movement 820,822relative to a display screen) that repositions one or more Nodes 803-810from a first position to a second position. A Vector 826 may begenerated at an angle that is perpendicular 825 or some other designatedangle in relation to the Smart Device 801. In some embodiments, an anglein relation to the Smart Device is perpendicular 825 and therebyviewable via a forward-looking camera on the Smart Device.

A user may position the Smart Device 801 such that an object in adirection of interest is within in the camera view. The Smart Device maythen be moved to reposition one or more of the Nodes 803-810 from afirst position to a second position and thereby capture the direction ofinterest via a generation of a Vector in the direction of interest.

Referring now to FIG. 8C, as illustrated, a Vector 825 indicative of adirection of interest may be based upon a rocking motion 823-824 of theSmart Device 801, such as a movement of an upper edge 818 in a forwardarcuate movement 823. The lower edge 819 may also be moved in acomplementary arcuate movement 824 or remain stationary. The movement ofone or both the upper edge 818-819 also results in movement of one ormore Nodes 803-810. The movement of the Nodes 803-810 will be asufficient distance to register to geospatial positions based uponwireless transmissions. A required distance will be contingent upon atype of wireless transmission referenced to calculate the movement. Forexample, an infrared beam may require less distance than a Wi-Fi signal,and a Wi-Fi transmission may require less distance than a cell towertransmission which in turn may require less distance than a GPS signal.In some embodiments, as discussed further below, hybrid triangulationmay include one or more distances based upon wireless transmissions ofdifferent bandwidths or modalities. For example, a first modality mayinclude Wi-Fi transmissions and a second modality may include Bluetoothtransmissions, still another modality may include infrared or ultrasonicmodalities.

Referring to FIG. 8D, line segments 831-838 are illustrated thatintersect various generated position points (PP1-PP8) for Transceivers803-810. Position points PP1-PP8 may be generated according to themethods and apparatus presented herein, including a mathematicalaverage, median, weighted average, or other calculation of multiplepositions determined via triangulation techniques. In addition, a Vector839 or Ray may be generated based upon one or more of the lines 831-838.In some embodiments, position points may be recorded in high numbersbased upon thousands of logical communications per second and a virtualrepresentation of the position points PP1-PP8 may be generated basedupon the recorded position points PP1-PP8. Some embodiments may alsoinclude a cloud point type representation a device that comprises theTransceivers used to record position point PP1-PP8, wherein the cloudpoint representation is based upon the multiple positions calculated.

Directional Wireless Modalities

Some modalities, such as those modalities that adhere to the Bluetooth5.1 or BL5.1 standards, allow a Node to determine an angle of arrival(AoA) or an angle of departure (AoD) for a wireless transmission. Anarray of antennas may be used to measure aspects of the Bluetoothsignaling that may be useful to calculate these AoA and AoD parameters.By calibrating an antenna system, the system may be used to determineangles in one or two dimensions depending on the design of the antenna.The result may be significant improvement in pinpointing the location oforigin of a signal.

An array of antennas may be positioned relative to each other and atransmitting transceiver to allow for extraction of an AoA/AoD. Such anarray may include a rectangular array; a polar or circular array; alinear array; and a patterned array, where a number of antennas aredeployed in a pattern conducive to a particular environment fortransceiving. Antennas may be separated by characterized distances fromeach other, and in some examples, a training protocol for the antennaarray results in antenna positioning incorporating superior angle andlocation precision. Some Nodes may transceive in 2.4-2.482 GHz frequencybands, and thus the RF transmissions may have wavelengths in the roughly125 mm length scale. A collection of antennas separated by significantlyless than the wavelength may function by comparing a phase of RFtransmissions arriving at the antennas. An accurate extraction of phasedifferences can yield a difference in path length that when accumulatedcan lead to a solution for the angles involved.

Referring to FIGS. 9A-D a series of exemplary devices employing matricesof antennas for use with Nodes that communicate via a Bluetoothstandard, a Wi-Fi standard or other modality, is illustrated. Linearantenna arrays 910 are illustrated in FIG. 9A. Rectangular antennaarrays 920 are illustrated in FIG. 9B. Rectangular antenna arrays 930are illustrated in FIG. 9C.

Nodes may include antenna arrays combined with batteries and circuitryto form complete self-contained devices. The Nodes or a controller maydetermine an AoA and/or AoD or other related angular determinationsbased upon values for variables involved in wireless communications. Inan example, a composite device 940 may be formed when a Node 942 with acircular configuration of antenna elements 943 is attached to anexemplary Smart Device 941. The Node 942 attached to the Smart Device941 may communicate information from and to the Smart Device 941including calculated results received from or about another Node, suchas a Node fixed as a Reference Point Transceiver or a Node with dynamiclocations, wherein the wireless communications are conducive togeneration of reference angles of transmission and/or receiving.

Referring to FIG. 10A, a Smart Device 1020 may be equipped with a Node1030 that includes a self-contained Bluetooth 5.1 antenna matrix. In theexample, the matrix of antennas in the Node 1030 may be configured in acircular pattern. Electronics in the device may capture communicationsignals sent from a wireless access point 1010. Each of the paths fromthe wireless access point to the various antennas of the Node 1030 has aslightly different path through air from the wireless access point 1010to the Smart Device. This may give each of the signals a slightlydifferent phase alignment with each other. The electronics of the Node1030 may include both hardware and software along, with training historyof the antenna array for the device and may be able to use the differentphase measurements and training history to determine both an azimuthalangle 1040 and altitude angle 1050 as an example. The resultingdirection pinpoints a significantly improved understanding of thelocation of the Smart Device 1020. In some examples, the calculatedresult may localize the Smart Device 1020 relative to the wirelessaccess point with an accuracy better than 50 cm. In desirable noise andsignal situations, a relative localization accuracy may be as good orbetter than 50 cm-level accuracy.

Referring to FIG. 10B, a combination of antenna arrays and electronicsto determine the angle of arrival or angle of departure may be placed inproximity to the Smart Device. In some examples, a combination of two ormore antenna array devices 1021 may be configured to independently sitin a plane proximate to the Node 1030, such as a Smart Device. Theantenna arrays may interact with two or more wireless access points 1010and 1060 which may also be called locators. When the multiple Rays arecalculated from each of the locators 1010 and 1060 to each of theantenna array devices 1021 a set of positional points for the twoantenna array device may result. These positions may again be used tocalculate a Ray 1070 of direction between the two points. This Ray mayrepresent the direction that the Smart Device is positioned in at aparticular time.

More complex combinations of the arrays of antennas may be configured toincrease the signal to noise of the system and improve accuracy. In anon-limiting example, three arrays of antennas 1021A, 1021B, and 10210,may be found in referencing FIG. 10C. In some examples, the size of theantenna devices may be such that a combination of them may be largerthan a Smart Device that they are associated with. In some examples,such as the illustrated example in FIG. 10C, the arrays of antennas1021A, 1021B, and 1021C may be overlapped in space. The result mayphysically relate to multiple overlapped regions of the antennaStructure. The resulting interaction of the Structures may be verycomplex, and training of the algorithms to extract results from thesignals received by the complicated Structure may be required to achievea directional result. The integration of multiple Structures can improvesignal-to-noise ratios related to transmission or reception of signalsin some examples; however, as the multiple results can be averaged (insome embodiments, a weighted average) to extract a direction of theorientation of the Smart Device.

Referring now to FIG. 11 , method steps are illustrated that may bepracticed in some embodiments of the present invention. At step 11, aunique identifier is established for each Node to be included in aself-verifying array. The unique identifier may be an alphanumericstring that is unique to available Nodes, a characteristic variable of asignal (e.g., characteristic frequency or wavelength), a public-keyencryption scheme, or any similar unique identifier.

At step 12, each Node (Node X) communicates with each other Node (NodeX+Y) with which Node X may establish wireless communications.

At step 13, sets of values for variables descriptive of respectivewireless communications are generated. Variables may include, forexample: which Nodes are involved in a wireless communication (which maybe determined for example via a unique Node identifier); timing valuesrelated to time of transmission of a data packet; timing values relatedto a time of arrival of a data packet; an angle of arrival of a wirelesstransmission; an angle of departure of a wireless transmission; astrength of a received wireless communication; a quality of a receivedwireless communication; or other variable. Each Node may generate a setof values for the variables for each wireless communication.

At step 14, optionally, each Node may record aspects of a wirelesscommunication that may influence accuracy of one or more values forvariables descriptive of respective wireless communications betweenNodes. Example of such aspects may include the presence of anobstruction to transmission of wireless communications, a strength of areceived transmission (for example a weak strength of a receivedtransmission may indicate a significant distance between the Nodes incommunication), and the like.

At step 15, each Node may store sets of values for the variables forrespective communications and aspects that may influence accuracy of thesets of values. In sonic embodiments, this step is optional; a Node maybe capable of immediately retransmitting a value for a variable withoutfirst storing it. In some embodiments, a Node may perform certaincomputations relating to the values for the variables, such as taking aweighted average of values received through multiple modalities orSensors.

At step 16, respective Nodes transmit respective sets of values for thevariables for respective communications and aspects that may influenceaccuracy of the sets of values to any other Node with wirelesscommunication range. In some embodiments, a Node may also transmit thesets of values for the variables for respective communications andaspects that may influence accuracy of the sets of values via hardwirecommunication.

At step 17, each Node within communication range receives thetransmitted sets of values for the variables and aspects that mayinfluence accuracy of values. By the process of generating sets ofvalues for variables of communications, receiving sets of values ofvariables for communications, and transmitting the same values, eachNode may acquire multiple sets of values relating to itself and to otherNodes, even Nodes that are out of range for direct communication and/orobstructed from direct communication. The multiple sets of values may beused to verify each other. In some embodiments, sets of outlier valuesmay be disregarded.

At step 18, using a controller with a processor and executable software,a position of a particular Node (X) may be generated based upon acomposite of sets of values, or a mathematical algorithm involvingmultiple sets of values. In addition, aspects that may influence thesets of variables may be given mathematical weight in generating aposition of Node (X).

At step 19, in some embodiments, a presence of an obstruction may beinferred based upon the multiple sets of values for variables incommunication. Still further a position of the perceived obstruction maybe generated based upon the same multiple sets of values for variablesin communication.

At step 20, a visual representation of a verified location for each Nodeincluded in the array may be generated, and in some embodiments, thevisual representation may include a position of a perceived obstruction.Each location is verified by sets of values for variables incommunications between multiple Nodes. Using this process, a position ofa Node may be made available to a Smart Device or another Node that isnot within direct communication range and/or is obstructed from directtransmission. Each Node generates values of variables for communicationthat may be used to determine a particular Node's position relative toother Nodes and/or a base position.

Referring now to FIG. 11A, a Structure space 1100 having a multitude ofwireless Nodes 1102-1106 is illustrated. Nodes 1102-1106 are shownlocated within or proximate to Structure space 1100. Nodes 1102-1106include Transceivers operative to communicate via a wireless modality,such as one or more of: Bluetooth 5.1; BLE5.1; Wi-Fi RTT, infrared, andultrasonic communications. In some examples, Nodes 1102-1106 includecomponents capable of supporting positioning and data transfer functionsuseful in establishing a self-verifying array of Nodes (i.e., a SVAN).

Nodes 1102-1106 may establish a self-verifying array 1117 with directcommunication paths 1110-1115 between Nodes illustrated by the dashedlines between the Nodes 1102-1106 positioned at disparate locations.Nodes that are within direct communication range are shown formingdirect communication connections along the direct communications paths1110-1115. Communications between Nodes include data useful fordetermining one or both of: a position relative to each other; and aposition of a Node to a base position 1116. Direct communications withinthe self-verifying array may also provide improved signal to noiseratios. In some embodiments, Sensors may be co-located with one or moreNodes and in logical communication with the Nodes, thus allowingtransmission of Sensor data across the Nodes.

According to the present invention, the self-verifying array 1117enables overall separations of Nodes that are larger than the directcommunication range of the individual Nodes 1102-1106. In other words,self-verifying array 1117 may allow a single Node to transmit tolocations greater than the Node's own transmission limits using otherNodes in the self-verifying array. For example, Node 1102 and Node 1105may not be within a direct communication range of each other due to thedistance D1 between Node 1102 and Node 1105 exceeding a range supportedby a modality of communication used by Node 1102 and Node 1105. However,data generated at Sensor 1105A that is co-located with Node 1105 may betransmitted to Node 1104 and then to Node 1103 and then to Node 1101;alternatively, and/or in addition, data generated at Sensor 1105A may betransmitted to Node 1106 and then to Node 1102, thereby extending thecommunications range of the modality in use.

In addition to Sensor data, values for variables of communicationsbetween various Nodes 1102-1106 may be transmitted amongst each Node1102-1105, where the values may enable a determination of a relativeposition of respective Nodes 1102-1105 to each other and/or to a baseposition 1116. In this manner, a position of any two Nodes 1102-1105relative to each other and/or to the base position 1116 may begenerated. Verification of Node 1102-1105 positions is accomplished viageneration of a particular Node 1102-1105 in relation to another Node1102-1105 and/or a base position 1116 using multiple sets of values ofvariables involved in disparate communications between disparate Nodes1102-1105.

In an example, the Structure space 1100 may be considered a Bluetootharena which is covered by a collection of Nodes 1102-1105 operative tocommunicate with at least the BLE5.1 standard and thereby form aself-verifying array, such as self-verifying array 1117. In theStructure space 1100, the self-verifying array 1117 may establish a baseposition 1116 from which positions of the various Nodes 1102-1105 may berepresented.

In some examples, the base position 1116 may be a spatially significantfeature such as a corner, door threshold, physically marked space, orthe like, which is established in a model sense with Nodes 1102-1105including Bluetooth Transceivers that are fixed within the space 1100.In other examples, the base position 1116 may be established at one ofthe stationary Node 1102-1105 locations.

Referring again to FIG. 11A, one exemplary Node (such as Smart DeviceNode 1102) may include an Agent-supported Smart Device. The Smart DeviceNode 1102 may be located at a fixed position and may serve as the baseposition. In some examples, the Smart Device may be a pad or touchscreen which is mounted to a wall position, or it may be a Kiosk-typedevice also located in a fixed position.

In other examples, a fixed Node 1103 may be located within the Structurespace 1100 such as at a ceiling-mounted position. Here too, this Node1103 may be established as the base position for Nodes 1102, 1104-1105across the network. In other examples, a base position 1116 may be at alocation offset from a physical, spatially significant architecturalfeature such as a corner of a Structure or a doorway.

An Agent supporting a Smart Device 1107 with a Bluetooth transmitter mayenter the Structure space 1100 containing the self-verifying array 1117and act as a Node in the self-verifying array. The various positioningcapabilities of the various Nodes 1102-1105 in the space 1100 mayactivate to provide location-positioning data to the Agent-supportedSmart Device 1107. In some examples, a base position unit is swapped tothe Agent-supported Smart Device 1107, in which case, all positions maybe dynamically updated relative to the Agent-supported Smart device1107. In some examples, multiple (and in some cases temporary)additional coordinate systems may be established in addition to a basedefinition of coordinate system which may have a fixed base unit.Exemplary coordinate designations may include Cartesian Coordinates,Polar Coordinates, Spherical Coordinates, and Cylindrical Coordinates,wherein Bluetooth-type designations of AoD and/or AoA and radius may berepresented as coordinates in a Polar, Spherical, or CylindricalCoordinate system.

There may be Nodes 1104 that are located upon equipment or appliances1104A and may therefore be stationary in most cases. The Node 1104co-located with the appliance 1104A may be powered by the appliancepower supply and also have battery-backup aspects. In the exampleillustrated, the Node 1104 on the appliance 1104A may be classified asthe base unit. However, as illustrated, it may be located at a remoteposition from a doorway to the space. Thus, the use of a self-verifyingarray may allow for a remote Smart Device 1107 to be au active Node inthe space 1100.

There may be Nodes 1105 that are located on wall buttons or inwall-positioned devices. Here too, such a device may be defined as thebase position unit. Such a device may be battery-powered and may requiremeans of battery replacement or charging. In some examples, the Node1105 may have a connection to utility power 1109 and or data conduit.The use of self-verifying array may allow for a User device (not shown)to be tied into a network that connects to the self-verifying array 1117that covers the bulk of the area of the Structure space 1100.

In some examples, a region 1106A of the Structure space 1101 may begenerally devoid of coverage to the self-verifying array 1117. Indesigning the communications environment of the space 1100, therefore, aNode 1106 with a Bluetooth transmitter may be fixedly located to aceiling, support pole, or other Structure feature in a region 1106A thatis otherwise devoid of communications coverage.

A visual representation of a self-verifying array may include some orall of the Nodes include in the array, and, in some embodiments, it willinclude a representation of a perceived obstruction based upon thevalues for communication variables. Some embodiments of a visualrepresentation may have one or both of layers of spatial grid definitionand polar coordinate definition. In a base layer, a coordinate systemfor the Structure space may be established using a fixed device as abase unit. The origin of this first layer's coordinate system may beestablished as a zero point in numerous coordinate system types such ascartesian, polar, cylindrical, spherical or other topographicalcoordinate models.

In some examples, an overlay second layer may include a coordinatesystem which is spatially similar to the self-verifying array, where forexample, each connection of three devices may create a regionalcoordinate system, and the Structure space 1100 is represented as amosaic of local coordinate systems within self-verifying-array-definedspaces. In some other examples, an overlay third or more layer may be adynamic coordinate system where a specific communication Node, which ismobile, is dynamically tracked as the coordinate system origin and therest of the space is adjusted relative to the moving origin.

Various embodiments may include schemes and layers of coordinate systemdefinitions that become defined for a composite of self-verifying arrayNodes 1102-1105. In some examples, one or more of the coordinate layersmay be defined, tracked, and communicated by a single network memberdefined as a base position unit. In other examples, a SVAN maydistribute coordinate definition and communication to Nodes 1102-1105dynamically. A routine update of calculated and measured position andcoordinate system may be maintained that not only defines a coordinatesystem but also indicates where some or all self-verifying arrayconnected Nodes 1102-1105 are located on a grid system. A routine updateat a schedule of time may therefore track Nodes 1102-1105 that aremoving in time, recalculating their position.

In some examples, a Bluetooth-enabled device may not be authorized ormay not have the capabilities to enter the self-verifying array as aNode 1102-1105, but it may emit signals including identificationinformation and may receive communications from the self-verifyingarray. In such embodiments, the self-verifying array may identify thesenon-Node-type communication devices and establish their positions. Aswill be defined in more detail, in some examples, a positiondetermination for a particular non-Node device may be defined inreference to a Node 1102-1105 that the non-Node device communicateswith, along with an estimate of a range in which the non-Node device iscapable of communicating.

Referring now to FIG. 11B, a Smart Device may receive a communicationfrom a Node 1102-1105 in a self-verifying array, wherein thecommunication includes multiple positional coordinates for each Nodeincluded in the array. The communication may also include positionalcoordinates of items of interest associated with Nodes 1102-1105 on thenetwork, such as an item co-located with a Node 1102-1105. In someexamples, the network may be interrogated by the Smart Device to provideinformation related to one or more Nodes included in the self-verifyingarray. The data may be used by the Smart Device to generate a userinterface 1121 with a pictorial/image representation of the variousBluetooth transmitters and network Node devices. The representation mayutilize one or more coordinate systems. For example, a Smart Device 1120may portray the user interface 1121 may include image representation ofa region of the Structure space which may be user selectable on theSmart Device. The image representation in the user interface 1121 mayinclude an origin 1122 designation for a particular coordinate layer. Inan example, where the coordinate layer is one where the origin isdynamically updated for the position of the Agent, then the origin mayrepresent the position at which the Agent is located.

In another example, an origin may be congruent with an origin of acoordinate layer for a spatially relevant origin and the Agent may ormay not be represented as an item on the user interface 1121. For thisexample, an Agent may be represented by position 1123. A pictorialrepresentation may show the Agent position 1123 and also presentparameters that refer to the Agent position such, as a two-dimensionalcartesian reference 1126 and/or a polar notation reference 1127. Otherwireless Nodes of relevance 1124 and 1125, within the scale of the imagemay be portrayed as well.

In some examples, the self-verifying array may include a feature wheresome or all of the network connected Nodes have identificationinformation associated with them. Each of the Nodes may have stored(locally or in another network data layer) a multitude of referencessuch as an identification information internal to a transceiver. Forexample, a Bluetooth transceiver may transmit identification informationlike a 48-bit Bluetooth transceiver address, a user-assignable name tothe transmitter, or a user-assignable name to an element that thetransmitter is a component of may be stored. As an example of anassignable name, in a non-limiting sense, an appliance may be a Node inself-verifying array which may have the name “Downstairs Refrigerator.”

In some examples, identification information may be related to differentlevels of security access that a Node may access, store, transmit, andthe like. Information useful for generation of a user interface may betransmitted from a Node via IP on a digital communications network, suchas the Internet, and a user may be located anywhere that is connected tothe communications network. In this manner, a user interface may bepresented to a remote user regardless of the user's location.

In some examples, a stable base unit of the self-verifying array may actas a standard repository and access point for all information stored orarchived for the self-verifying array. In other examples, the datastorage may be distributed across the self-verifying array. In anexample, a standard portion of the data stored on the self-verifyingarray, such as in a non-limiting sense, the identifications, timestamps,positions, characteristics, and security levels of all Nodes, andidentifications and positions of all transmitters within the Structurespace/self-verifying array extent may be assembled into a datatable/layer.

In some examples, a routine transmittal of a data table/layer may bebroadcast throughout the self-verifying array. In an example, everyself-verifying array Node may have an assigned broadcast order such thatat a standard time indexed to the broadcast order, it will broadcast itscurrent version of the table. All Nodes within range of a transmittingNode will receive the table and update it as the current version. Then,at their prescribed broadcast time, they might transmit the table. Theremay be rules that overlay such a broadcast to ensure that current datais not updated with previous versions for a Node that does not receivethe update before its broadcast time. Such rules may also preventunauthorized alteration of data through hacking or other networkpenetration. The Nodes may act as participants in a Blockchain in thismanner.

One such rule may be that transmission may occur only when the datatable has been updated at the Node. Another rule may inhibittransmission for any Node that is dynamic/moving, or alternativelyinitiate immediate transceiving for a Node that is dynamic/moving.Transmission may include diverse types of data. Periodic transmissionsmay be timed based upon a time needed for a transmission, energyrequired for transmission, available energy, receipt of new data, andthe like. Therefore, each Node may have a configuration setting thatdefines conditions when, how, and for how long it transceives. Suchcondition may include, for example, a frequency upon which it listensfor and upon which it communicates data. The various definitions ofcoordinate layers may be transmitted.

In some examples, a remote user-connected digital communications networkto a self-verifying array Node or a Bluetooth device entering into aself-verifying array Node may request a copy of a standard data tabletransmission. The data table transmission may include positions of Nodesrelative to a fixed origin, to the user position, to particular fixedNodes of the network or a collection of some or all of these.

Some data layers may be created to store Sensor information that may beobtained at some or all of the Nodes. The data layer may be segregatedbased on types of Sensor data. For example, all Nodes of aself-verifying array may include a Sensor providing a quantification ofone or more of: ambient temperature, humidity, water presence, currentdraw, vibration, movement, image data, and the like. A timestampedreading of this Sensor quantification may be included into a data layeralong with co-located Node identification information.

In other examples, a subset of the devices may include an ambient-lightSensor as part of its infrastructure. In this case, another data layermay be created for this type of Sensor data. In some examples, thepictorial image representation 1121 may include one or more of the datalayer Sensor information. The pictorial image representation 1121 mayrepresent the Sensor readings in a textual form, or in other mannerssuch as a color indication at a Node position or at regions around aNode position.

Referring to FIG. 12A, another representation of a SVAN 1210 isdisplayed. In this embodiment, space 1200 may include Structures 1211and 1212. Structures 1211 and 1212 may have a variety of differentcharacteristics that may impact the performance of self-verifying array1210. For example, Structures 1211 and 1212 may be physically closed(e.g., walls, solid Structures) or partially closed (e.g., shelves).Structures 1211 and 1212 may also comprise solid materials (which may bestored for example at a construction site), equipment such asautomobiles, tractors, backhoes, and the like. Accordingly, the presenceof these Structures may change the transmission characteristics of awireless network (e.g., Bluetooth). Some Structures may block signals,impede signals, or partially impede signals. For example, shelves mayhave physical regions that block and other regions that are fullytransmissive.

Shelves may provide an example in which the Structures in the space 1200may have dynamic characteristics. Such dynamic characteristics may makeself-verifying arrays (and corresponding spatial schema) more usefulthan traditional mapping methods. For example, if a load of metal pipingis brought in and placed upon the shelves, a region that was completelytransmissive may become impeded to a degree. These characteristics maycreate different operational characteristics for self-verifying arrays.

In another sense, a shelf may hold a combination of both fixed andmobile devices that comprise a self-verifying array in the space at somegiven time. This may provide more accurate and more dynamic coverage forthe schema. For example, the space 1200 may be interspersed with anassembly of fixed (or roughly fixed) network Nodes that form a gridpattern (as an example) to ensure that a minimal self-verifying arraymay be established that covers the entire space 1200. This minimalnetwork may be supplemented with “migrant” Nodes that are moved into thespace 1200 and become part of the SVAN 1210. From a signal coverageperspective, more participants may improve characteristics. However,more participants may increase information traffic levels, and a controlformalism that limits bandwidth differentially to different networkparticipants may be necessary in some examples.

In FIG. 12A, an example of a space 1200 with shelving units that make upStructures 1211 and 1212 is illustrated. The space may have a “global”reference point 1204 for positioning. There may be fixed wirelesscommunication Nodes 1201, 1202, and 1203 (for this example, all Nodesare at least compliant with Bluetooth 5.1 and transmit at least asBluetooth radio transmitters; however, this deployment is merelyillustrative). The fixed wireless communication Nodes 1201-1203 may alsoinclude other aspects/components to them such as an integrated camerasystem. The integrated camera system may provide a visual perspective ofa portion of the space that its corresponding wireless radios may cover.In a self-verifying array, Nodes may be collocated or located relativeto a Sensor, such as an image-capture device. Based on a known setposition of the Sensor relative to the Node, the Node may transmitinformation captured by the Sensor to other Nodes. Accordingly, a Nodeout of both Sensor and radio range of another Node may still receivedata from the Sensor through the array. The data from the Sensorreflects a range of data in which a physical characteristic isquantified or capable of being quantified by the Sensor. For example, aSensor may be an image-capture device, limited in range by bothwavelength of image capture (e.g., limited to infrared) and spatialrange (e.g., field of view of the image-capture device). This may beparticularly desirable in embodiments in which the self-verifying arrayis deployed in or adjacent to an environment having a characteristicadverse to a Sensor. For example, the low temperatures found in acommercial freezer may impair operation of certain Sensors.Temperature-resistant Sensors may be collocated with Nodes within thefreezer, while temperature-vulnerable Sensors (including Sensors capableof detecting conditions within the freezer) may be collocated outsidethe freezer. Through the self-verifying array comprised of these Nodes,data from the Sensors may be freely transferred among the Nodes,including through fiber optic communication throughout the freezer. Itmay be desirable to deploy spectrometers and hydrometers in thisfashion. Moreover, redundant Nodes may be able to redirect Sensorreadings from one Node to a base Node, especially in scenarios when anoptimal Node pathway may be obstructed, such as by shelving.

The space 1200 may also include other fixed Nodes, such as Node 1223,that may not have cameras included. Node 1223 may be important to ensurethat regardless of a makeup of migrant communication Nodes, fixedwireless communication Nodes may be able to form a complete SVAN space1200 in the absence of items that block radio transmissions. There mayalso be migrant communication Nodes 1220-1222 affixed on packages,materials, or other items that may be placed and/or stored upon theshelving units.

In some examples, at least a subset of the SVAN-participant Nodes maycommunicate periodically. The various aspects of data layercommunications as have been discussed may occur between the Nodes of thenetwork. At a base level, at least a subset of the Bluetoothtransmitters may periodically transmit information such as their uniqueidentifiers, time stamps, known positions and the like. In someembodiments, Nodes may transmit between each other or to a baseinformation about variables between the Nodes, such as computeddistances or angles between the Nodes. A Node may receive transmissionsfrom other transmitters and may store the transmissions. In someexamples, a Node may act as a repeater by receiving a transmission andthen retransmitting the received transmission. A Node acting as arepeater may then take various actions related to the data involved. Inan example, the Node may effectively just stream the data where nostorage of any kind is made. Alternatively, a Node may store thetransmission, then retransmit the transmission (immediately or after adelay) and then delete the stored data. In other examples, a repeaterNode may store a received transmission and then retransmit thetransmission either for a stated number of times, or until some kind ofsignal is received after a transmission. Thereafter the Node may alsodelete the data. In some examples, a Node may store data from anincoming transmission and take the various retransmission actions ashave been defined, but then not delete data until its data store isfilled. At that point, it may either delete some or all of the storeddata, or it may just overwrite stored data with new incoming data andthen clean up any remaining data with a deletion or other process.

When a Node acts as a repeater, it may receive data and then merelyretransmit the data. Alternatively, a repeater Node may either use thetransmission of data or the time during the transmission to acquire andcalculate its position and potentially the position of othertransmitters in range. During retransmission of the received data, itmay also include in the transmission calculations of its own positionrelative to other transmitters, calculations of other transmitterpositions relative to itself, calculations of its own and othertransmitter positions relative to an origin, and the like. It may alsoinclude other information such as a time stamp for the calculation ofpositions.

The combined elements of a SVAN may be operated in a way to optimizepower management. Some of the network Nodes and transmitting elementsmay operate in connection with power-providing utility connections inthe Structure. Other network Nodes may operate on battery power. Each ofthe Nodes may self-identify its power source, and either at a decisionof a centralized controller or by a cooperative decision-making process,optimized decisions may be taken relative to data transmission,low-power operational modes, data storage and the like. In someexamples, where multiple Nodes provide redundant coverage and provideinformation to a central bridge acting as a repeater, the Nodes mayalternate in operation to share the power-draw on individual Nodes. Forexample, if one of these Nodes is connected to a utility power source,that Node may take the full load. The battery-powered elements may havecharge-level detectors and may be able to communicate theirpower-storage level through the network. Accordingly, an optimizationmay reduce traffic on the lower battery capacity Nodes.

In some examples of operations, a transmitting Node may transmit amessage for a number of redundant cycles to ensure that receivers have achance to detect the message and receive it. In low power operatingenvironments, receivers may transmit acknowledgements that messages havebeen received. If a base unit of the network acknowledges receipt of themessage, control may be transferred to the base unit to ensure that themessage is received by all appropriate network members. Messagereceivers may make a position determination and broadcast their positionif it has changed. A self-verifying array of Bluetooth receivers mayprovide one of a number of Transceiver network layers where othercommunication protocols based on different standards or frequencies ormodalities of transmission may be employed, such as Wi-Fi, UWB, Cellularbandwidth, ultrasonic, infrared and the like. A Node that is a member ofdifferent network layers may communicate and receive data between thedifferent network layers in addition to communicating through aBluetooth low-energy self-verifying array.

Referring to FIG. 12B, an illustration of the view from a camera on anetwork Node position is presented. A Smart Device 1250 may interactwith the self-verifying may and communicate a desire to receive imagesor video from a camera. In an example, referring back to FIG. 12A, theNode 1201 may have a camera that produces an image that in FIG. 12B ispresented on the smart phone as image 1260. Processing either on theSmart Device or on processors connected to the network may collectinformation about the location of other network Nodes through thevarious processes as described herein and then determine a correctlocation on the collected image to display icons over the position ofthe Nodes 1221 and 1223. There may be numerous other types ofinformation that may be overlaid onto the imagery such as Sensormeasurements, textual presentations of data values, data related tostatus and transactions on the network, and the like.

In some examples, the cameras may be maintained in a fixed position orpositioned on mounts that can allow the plane of view of the camera tobe moved. The Smart Device 1250 may be supported by an agent such thatit is oriented in such a manner to point to a particular view-plane fromthe perspective of the screen. This may either be from a perspective oflooking through the smart screen (i.e., in the field of view of a cameraassociated with the Smart Device 1250) or, in other examples, supportinga screen of a Smart Device flat (i.e., parallel to a ground plane) andpointing in a direction of interest based on a direction of orientationof the Smart Device 1250. In related applications, it is disclosed thata direction of interest may be determined based on wirelesscommunications. In some examples, orientation aspects of Transceiversupon the Smart Device 1250 may be employed to determine Rays of interestof the user (for example, to point the Smart Device 1250 in a directionof interest to the user). In other examples, other Sensors on the SmartDevice such as accelerometers, magnetometers, and the like may be ableto infer a direction in which the Smart Device is pointed. Once adirection of interest is determined, the camera may be moved tocorrespond to a plane perpendicular to a Ray associated with thedirection of interest (i.e., such that the Ray is a normal vector to theplane). An assessment of items of interest whose coordinates may lie inthe field of view of the selected view plane may be made, thuspresenting data to the user and allowing the user to filter out or learnmore about the items.

Referring now to FIG. 12C, another type of presentation is illustratedwhere a plan view or map view of the space 1270 may be presented. Insome examples, a Smart Device may access a virtual model (AVM) or otherspatial schema that contains spatial data relating to the space that theuser is in. The view may also include a presentation of the Structure,including features such as walls, doors, supports, shelving, equipmentand the like. The location of network Nodes may be illustrated by icons1273 at the two-dimensional locations determined by the variousposition-mapping processes as described herein. The location of the user1271 may also be superimposed upon the map with a different icon, andthis location may be dynamically updated as the user 1271 is moving.There may also be an iconic representation of the heading 1272 of theuser which may be determined by the wireless protocols as discussedherein or it may be estimated based on the time-evolution of theposition of the user (for example, through dead-reckoning). Items ofinterest may be presented on the map at any location surrounding theuser such as in front, to the side or behind the user. In some otherexamples, only items in the view-plane (determined by the heading of theuser) may be illustrated on the Smart Device 1250. Textual data andother types of information display such as color gradation may also besuperimposed on the map to represent data values such as Sensor input,network characteristics, and the like. In some examples, a relativeheight of items of interest relative to the floor or to the Smart Devicemay be presented on the image as a text presentation.

Referring to FIG. 12D an extension of location tracking is illustratedfor devices that do not have positional capabilities (such as a GPS) butcan respond to transmissions within a certain distance. The range of thedevice can allow a localization to a be within a certain distance from aNode. In some examples, nanowatt Bluetooth Nodes that operate withoutbattery power may be cheaply attached to items for tracking and/or canbe affixed with Sensors to provide data acquisition. These devices maytypically depend upon energy harvesting for their operation. In someexamples, a transmission from a Node of the SVAN may itself carry enoughenergy to enable an RFID tag or other type of passive tag to respondwith a low-energy transmission. Accordingly, a Node may transmitsufficient energy to activate an RFID; such as, for example, an RFIDthat has an identifier of an item to which it is affixed. The devicesmay be unable to perform all the wireless functions discussed herein,but they may be capable of transmitting identification data and perhapsSensor data.

In some examples, RFIDs may be employed. Bluetooth self-verifying arrayNodes may also have incorporated RFID tag readers that can similarlytransmit a unique identifier in response to a transmission from theself-verifying array Node. In FIG. 12D, a Smart Device 1250 may displaya map-form presentation of a space 1270 (similar to the previousdiscussion with SVAN Nodes located in a two-dimensional coordinatesystem). In an exemplary embodiment, ultralow-power Bluetooth Nodes orRFID Nodes may be located on elements such as packages or equipmentplaced on the illustrated shelves. In response to transmissions from theSVAN Nodes, various low-power tags may respond. In some examples, thelocalization of the low-power tag may be based on further refinement ofmeasurements, such as measurements of the returned signal strength.

Referring again to FIG. 12D, a SVAN Node 1273 may detect twotransmitting Nodes (labeled “A” 1280 and “B” 1281 in FIG. 12D). SinceNode “B” 1281 may also be detected by a neighboring SVAN Node 1274, itmay be inferred that the Node may be in a region located between the twoSVAN Nodes (i.e., since Node “B” is located in the overlap of thecoverage areas of SVAN Nodes 1273 and 1274, it is likely that Node “B”is located somewhere between SVAN Nodes 1273 and 1274). Other Nodesreceived by SVAN Node 1274, such as Nodes 1282 and 1284, may not bedetected by other SVAN Nodes and thus may be located in non-overlappedregions. As a further illustration, Node “D” 1283 may be detected bothby Nodes 1274 and 1275. Node “F” 1285 may be detected by three differentSVAN Nodes 1274, 1275 and 1276. Thus, the position of Node “F” may bedetermined with a high level of confidence. Node “G” 1286 may be locatedonly in SVAN Node 1276. Node “H” 1287 may be located only in SVAN Node1275. This data may provide localization information for regions aroundBluetooth SVAN Nodes.

The non-limiting example discussed has included a Structure withobstructions in the form of shelves; however, obstructions may includeany item that interferes with or inhibits or decreases quality ofinter-Node communication within the self-verifying array.

Some self-verifying arrays may be established in an outdoor environment,such as a construction site. There may be numerous items, such asequipment, tools, and materials to which Nodes may be attached. In someexamples, at a construction site there may be significant utility inestablishing fixed Transceiver position as the site is initiallyestablished. The self-verifying array may track and locate the variousequipment and materials through radio-frequency communications (e.g.,via RFID). Furthermore, establishment of fixed points across theconstruction site may allow for a self-verifying array of significantsize to be established. As described in reference to FIG. 12D, there canbe RFIDs or Bluetooth Nodes that may be attached to various materialssuch as structural beams, wallboard pallets, and the like. In theseexamples, the transmitting Nodes may not have battery elements for costor environmental conditions reasons. The location of various componentsof construction may be tracked as they are moved across the site. Insome embodiments, an AVM may be used to compare expected movements ofcomponents to the observed movements. As the Structure is built andstudied during the creation of AVMs, the various Bluetooth Nodes maystill be able to provide communications as to components that make upStructure or are embedded within Structure.

Referring to FIG. 12E, elements of a self-verifying array in a space1270 may have dynamic locations and their movement may haveramification. In an example, SVAN Node 1276 may physically move toanother location. The various self-verifying array data layers relatingto location of elements may update for this move and the updated tablesmay be communicated to the Nodes of the network as has been described.At the new location, SVAN Node 1276 may signal to devices in its newregion for response. There may be transmitting Nodes and RFIDS that areand have been in the new region that SVAN Node 1276 has moved to. Forexample, item “I” 1294 may be located by SVAN Node 1276 in its newlocation. As well, items with transmitting Nodes on them may also moveas illustrated by the detected movement of item “D” 1283. Another typeof change may be that when Node 1276 occupies its new location, item “H”1287 may be detected in the region of two network Nodes now, andtherefore its location may be refined to that region that the twonetwork Nodes overlap in coverage.

Referring now to FIG. 12F, an illustration of a complex space whereregions within the space may block or impede wireless communications isprovided. In some examples, parts of a Structure like internal walls,conduits, equipment, structural beams, and elevators/shafts may providepermanent or temporary blockage of wireless transmission. For example,as an elevator passes through a particular floor, it may blocktransmissions through the elevator shaft that may otherwise occur.Shelves may temporarily have materials or equipment moved to positionson the shelves as illustrated by regions 1297 and 1298, which may blockwireless transmissions. The self-verifying array 1200 and its Nodes1201-1203 and 1220-1223 may be able to cooperate and provide coverage ofthe self-verifying array around such blockage. For example, a wirelesscommunication Node 1296 may be too far from Node 1203 to communicatedirectly with it and communication from other fixed Nodes like 1201 and1202 may be blocked by the blockage as discussed. The SVAN may stillcommunicate 1295 with the SVAN Node 1296 by connecting a path 1299 shownin thick dashed lines essentially communicating with line-of-sight pathsaround the blockages.

Referring to FIG. 13A, mobile elements such as UAV and UGV with wirelesstransmitting Nodes attached are illustrated. Mobile elements mayfunction within self-verifying arrays to create dynamic physicalextensions of the self-verifying array. The mobile elements 1310,1320and 1330 are illustrated as UAVs. As the mobile elements move, they mayallow other Nodes or wireless access Nodes to make communications. Insome examples, there may be at least a first fixed element 1300 that ispart of the SVAN. It may define an origin point in some systems, but inother examples, it may be offset from an origin point 1310. As a mobileelement 1320 moves through space, its position may be updated bycommunication between the fixed element 1300 and itself 1320. Thelocation determine may in some examples be referenced to the origin. Inpolar notation, it may be located at r₂,θ₂, for example, where theangular components are taken with respect to an axis having at least apoint perpendicular to mobile element 1320 (e.g., a ground plane).

When the mobile elements are able to communicate with a fixed element, adetermination of the fixed element's position relative to a localcoordinate system may be straightforward since the fixed element canknow its position with relatively high accuracy. A moving device thatcan continually measure its position relative to the fixed element cancome close to that accuracy in position determination as well and canimprove its determination by taking more measurements. As mentionedpreviously, elements in an operating space may be either statically ordynamically positioned and block or impede wireless transmission throughthem. Mobile communication elements create interesting solutions in suchan environment because a team of communication elements can positionitself in such a manner to “look around” such difficult transmissionzones. At the same time, the difficult transmission zone may block theability of a mobile element from communicating directly with a fixedcommunication Node. In such cases, a first mobile element may determineits position relative to a second mobile element, where the secondmobile element has communication capability with fixed self-verifyingarray communication Nodes. In some examples, a self-verifying array mayconsist entirely of mobile elements, and then its practical coordinatesystem may be a local one that is determined in a moving coordinatesystem related to one or more of the mobile elements relative positions.

Referring now to FIG. 13B, an exemplary embodiment of thisnon-line-of-sight position detection is shown. In some examples, theremay be mobile elements 1350, 1351 with wireless communicationscapabilities that create at least a portion of a SVAN of wirelesscommunicating devices. In some examples, the wireless communicatingdevices may include capability for Bluetooth protocol communications. Instill further examples, the Bluetooth protocol communications devicesmay include capabilities for establishing self-verifying arrays as wellas capabilities of performing positioning based on AoA measurements suchas is defined in the Bluetooth 5.1 standard. A fixed element 1300 whichhas a known offset to position T₁ may locate a mobile Node 1350 (such asa UAV) at position T₂ in accordance with the orienteering methodsdescribed above. In some examples, the mobile Node 1350 at position T₂may have moved into position T₂ in order to have a line-of-sight withthe mobile element 1351 at position T₃. For illustration and discussion,the devices are shown with line-of-sight between T₁ and T₂ and betweenT₂ and T₃. In some examples, the wireless communication modalitiesdescribed herein may be capable of passing through walls or otherblockades, however, a blockade may resist or interfere with suchwireless transmission. In some examples, a wireless modality deployedmay just not be able to penetrate a given wall or other obstruction.

Accordingly, the second reference Transceiver T₂ of the mobile Node1350, due to its movement, may be deployed within the line of sight ofboth T₁ and T₃ to assist with determining an accurate location of T₃notwithstanding the lack of sight between T₁ and T₃. Although thisFigure shows a lack of line-of-sight between fixed device 1300 and themobile element 1351 as caused by blockade B 1370, line-of-sight may alsobe defeated by, for example, an excessive distance between T₁ and T₃(i.e., r₃ 1365). For example, Bluetooth 5.1 has a range of approximately1.4 km at certain frequencies. Thus, where r₃>>1.4 km, the presentmethod may be desirable for Transceivers that use Bluetooth 5.1.

Using the methods described above, the fixed element 1300 referenced toT₁ may determine the location of the mobile Node 1350 T₂ byline-of-sight communication. For example, the location may be determinedbased on the angle of arrival of signals, as angle θ₁ 1361 from T₂ andthe distance r₁ 1360 between T₁ and T₂ as measured by timing signals.For ease of calculations and discussion, the local coordinate system ofmobile Node 1350 at T₂ may be oriented to a reference direction 1352pointed to location T₁ from T₂. The mobile Node 1350 at T₂ may in turndetect the location of the mobile element 1351 at T₃, using (in someembodiments) the methods described herein. If T₂ uses the methodsdescribed herein to determine the location of T₃, it may determine thatthe mobile element 1351 at T₃ is located a distance r₂ 1362 from it andrelative to its reference direction 1352, it may be located at an angleθ₂ 1363.

The mobile Node 1350 may aid the system of SVAN elements to determinethe positions of each of the element relative to each other by relayingthe relative location of the mobile element 1351 at T₃ as detected tothe fixed element at 1300 which is referenced to the point T₁. One ofthe components of the SVAN, which may even include connected serversthat are connected to one of the self-verifying array Nodes, may thenperform algorithmic calculations to trigonometrically compute severaluseful values, such as: the effective distance between T₁ and T₃,notwithstanding blockade B 1370, i.e., r₃ 1365; the effective angle ofarrival of a signal from T₃, i.e., θ₃ 1366; the angle between T₃ and anaxis formed by Tr, i.e., θ₁₋₃ 1364; and the like.

Referring briefly to FIG. 15 , an exemplary method of computing thedistance between two nodes not having line-of-sight communicationsbetween each other is shown. In this example, it will be assumed thatthe nodes and the vectors between them can accurately be projected intoa two-dimensional, coplanar space, as shown in FIG. 15 . This may alsobe appropriate in situations in which, for example, three linearlyindependent axes can be determined (e.g., x, y, and z), but one of thoseaxes is not of interest. For example, if a flight path is to bedetermined in a warehouse, one might treat all blockades as having aheight equal to that of the entire warehouse, and then seek to avoidblockades on the x and y axes. In exemplary embodiments, the axes may betangible guides, such as a floor or a wall.

For the purposes of this discussion, let the distances between Nodes T₁and T₂, T₂ and and T₃ and T₁ be r₁, r₂, and r₃, respectively. Let theangles between r₃ and r₁, r₁ and r₂, and r₂ and r₃ be θ₁, θ₂, and θ₃,respectively. As described above, the magnitudes of r₁ and r₂ may beknown using the methods disclosed herein. The present invention alsoallows the position of T₃ to be communicated to T₁ using T₂ as a relayin a variety of ways; one exemplary way is as follows.

A straightforward way of computing the magnitude of r₃ is to use the lawof cosines. Doing so requires knowledge of at least θ₁, θ₂, or θ₃. θ₂can be determined in multiple ways, depending on the specifics of thedeployment of T₁, T₂, and T₃, as well as the specific Bluetooth 5.1implementations of each. For example, in some embodiments, θ₂ may merelybe any of the angle of arrival at T₂ or the angle of departure at T₂. Inembodiments in which a central controller effectively creates a map ofthe Nodes and translates them into a coordinate system, then 02 may bedetermined using a dot product or other norm between the vectorsrepresented by r₁ and r₂. In other embodiments, θ₂ may be determinedgeometrically as discussed in further detail below. In still otherembodiments (particularly those employing a central controller), thevector represented by r₂ may be translated to the origin (shown in FIG.15 as T₁) or otherwise measured to determine its magnitudes in each axisof the chosen coordinate system. This may then be used to determine themagnitude of r₃, as in the embodiment shown in FIG. 15 , r₃ is thevector sum of r₁ and r₂.

Assuming r₁, r₂, and θ₂ are known with accuracy, then the law of cosinesprovides that r₃ is simply equal to the positive square root of r₁ ²+r₂²−₂r₁r₂ cos(θ₂). (This computation may also be applicable inthree-dimensional models.) In practice, however, some or all of thesequantities may be subject to uncertainty. Accordingly, in someembodiments, several methods of computing r₃ (some of which arediscussed below) may be used, and a weighted average of thesecomputations may be taken to more accurately determine r₃. Moreover, themethods discussed below may produce additional quantities that may bedesirable in some embodiments, such as a virtual angle of arrival of asignal from node T₃ to node T₁.

In some embodiments, θ₂ may not be cleanly determinable as simply anangle of arrival/departure of a signal at T₂. However, in someembodiments, the angles of arrival/departure at T₂ may be determinedwith reference to an axis drawn parallel to the x axis, as shown indashed lines in the figure. Let these angles be φ₁ and φ₂. If φ₁ and φ₂are determined with accuracy, then theta₂ is 180°−φ₁−φ₂, and thecomputation of r₃ can proceed as discussed above.

Given r₃, other useful quantities may be computed. For example, althoughthe figure shows θ₃, it may not be immediately quantifiable as an angleof arrival/departure because θ₃ represents the angle between r₂ (i.e.,the vector connecting T₂ to T₃, the magnitude of which is known a prioriin some embodiments due to the line-of-sight tracking described herein)and r₃ (which is a virtual vector that has unknown characteristics apriori due to the lack of a line of sight between T₃ and T₁). But oncer₃ and θ₂ have been determined, then θ₃ is the arcsine of (r₁/r₃) sinθ₂. Similarly, θ₁ is the arcsine of (r₂/r₃) sin θ₂.

Referring back to FIG. 13B, analysis techniques, such asartificial-intelligence techniques, may also use a difference in aposition calculated trigonometrically and via delayed line-of-sight tocalculate an interference factor of a particular wall, material, etc.(such as blockade B). This may be used in subsequent transmissions thatmay pass through the particular wall, material, etc. to more accuratelyestimate the impact of the wall, material, etc. on the transmission.While the blockade B 1370 is stationary and static, it may be possibleto determine a calibration factor for signal changes caused by blockadeB 1370 that may allow for attenuated signals that come fromself-verifying array Nodes that are behind the blockade to none the lessbe directly estimated for their relative position.

In addition, a known delay can be used to determine attributes of anobstruction, such as material type, thickness, proximity, etc. This maybe particularly true when the blockade is uniform in itscharacteristics. Moreover, the trigonometric techniques discussed hereinmay assist in determining a lack of an obstruction between T₁ and T₂ ata given wavelength by comparing the expected trigonometric result withan empirically determined line-of-sight result.

It may be useful in controlling a particular space, such as aconstruction site, to utilize a team of mobile devices to survey andsurveil the space. In addition to the ability to surveil a region thathas regions of blocked/impeded transmission, the mobile network canestablish routine (but transitory) bridge links in a self-verifyingarray to communication Nodes that are remote, as has been described. Inaddition to these abilities, a mobile element that has an RFID readercapability may also pass over a space and “inventory” RFID tags attachedto items for security, location and condition tracking.

As mentioned previously, low-energy Bluetooth-based Nodes may also beinterrogated by mobile elements where these Nodes may also providesensing capabilities. As a non-limiting example, a construction site maybe modelled in an early version of an AVM for the Structure and it maytrack the location of components that will be assembled into theStructure as well as tools that may be used to construct the Structureas they arrive and, in some cases, leave a job site.

In some embodiments, a mobile Node is moved about to multiple locationswithin a physical area, such as during variations occurring during aconstruction job site. As the Node is moved, a height andtwo-dimensional coordinates of the mobile Node may be varied such thatit becomes possible for the mobile Node to communicate with other Nodesin or proximate to the physical area.

In some embodiments, the mobile Node may additionally communicate withother transceiver, such as a Bluetooth Node transmitter, an RFIDtransceiver, ultrasonic transceiver, infrared transceiver and the like.In some embodiments, the mobile Node may additionally transmit wirelessenergy to a receiving Node, RFID, or transmitter Node specifically toenergize the receiving Node, RFID, or transmitter Node, and enabletransceiving by the energy receiving Node, RFID, or transmitter Node.

Referring now to FIG. 14 , method steps that may be implemented in someembodiments of the present invention are illustrated. At method step1401, in some examples, a user may begin by installing wireless accesspoints into a building Structure as it is built. In other embodiments,the wireless access points may be added into the Structure after it isbuilt.

Continuing to method step 1402, a process may next be initiated that mayestablish a self-verifying array between the installed wireless accesspoints and other devices that are within the communications range of theinstalled wireless access points. Security protocols may control whethera particular communications element that is within range of such aself-verifying array may gain access.

Continuing to method step 1403, the self-verifying array may detect anentry of a wireless transmitter into an area covered by theself-verifying array. Entry may involve a physical movement of thewireless transmitter or the virtual movement of the coverage of theself-verifying array to include the wireless transmitter. Entry may alsoinclude reception of a previously unreceived signal from a wirelesstransmitter. In some embodiments, entry may include reception of apreviously unreceived frequency of a signal from a wireless transmitter.For example, it may be desirable not to detect the wireless transmitteruntil a chosen time, at which point a switch or other apparatus may varythe frequency of the signal from the wireless transmitter.

Depending on various security protocols and generalized networkprotocols, an optional method step at 1404 may be performed toincorporate a newly detected wireless transmitter (such as a mobiledevice) into communications with the self-verifying array. Proceeding toStep 1405, the network may optionally be configured by a user to directa movement of one of its mobile wireless access points into a newlocation while still maintaining its communications capabilities withthe self-verifying array. Proceeding to Step 1406, the network mayoptionally be configured by a user to direct a movement of one of itsmobile wireless access points to a location where it can simultaneouslybe connected to the self-verifying array while also establishingcommunications interchange with a device capable of wirelesscommunications where the device may otherwise not be in range with theself-verifying array.

Commercial Implementations of Self-Verifying Array of Nodes

Self-verifying arrays of Nodes are applicable in many diverse commercialimplementations. The following paragraphs describe several diverseimplementations utilizing a SVAN.

Referring now to FIG. 16 , method steps are illustrated for deploying aSVAN to quantify conditions in a parking area. The parking area mayinclude, for example, a garage or parking lot. Specific embodiments mayinclude one or more of: a rental car parking area; a commercial parkingarea; a residential parking area; a municipal parking area and the like.

At step 1601, a first Node may be fixedly attached to, placed inside of,or otherwise co-located with a vehicle. The Node will move with thevehicle as the vehicle moves.

At step 1602, a unique identifier associated with the first Node mayalso be associated with the vehicle with which the Node is co-located.For example, a database may store an association with the uniqueidentifier of the first Node with a Vehicle Identification Number (VIN)of the vehicle.

At step 1603, reference position Nodes other than the first Node may belocated at strategic placements within or proximate to the parking area.In some embodiments, the strategic placements selected for referencepoint Nodes may be based upon one or more of: a shape of parking area; awireless modality distance capability; a presence of obstacles within anarea occupied by a SVAN; at ends of rows defused for parking vehicles;at some or all defined parking spots for parking vehicles at one or morepoints of interest in a parking lot, such as a point of entry or egress,an office, a walkway, connecting transportation (railway or bus), anelevator or stairwell and the like.

At step 1604, one or more Nodes included in a SVAN may be designated asa Base Node. Base Nodes may be operative to perform functions notnecessarily performed by Nodes that are not Base Nodes. For example,Base Nodes may aggregate data over time, perform controller functions,transmit data via more than one wireless modality, be powered byutility-based alternating current, or communicate via a hardwired medium(e.g., via ethernet cabling).

At step 1605, one or more of the Nodes may communicate with other Nodes.Preferably, each Node will communicate with each other node within rangeof a communication Modality. In some embodiments, a pattern of Nodecommunication may be followed.

At step 1606, in some embodiments, a pattern of communication maystagger a time of wireless communication in order to avoid interferenceof one communication by another communication. A pattern ofcommunication may therefore include a “cascade” or hierarchical tree ofwireless communication transmission and receipt. For example, a BaseNode may communicate first, followed by a first generation of Nodes thatreceive a communication from the Base Node, and followed bycommunicating from the first generation of Nodes with a secondgeneration of Nodes (e.g., Nodes that are out of range or obstructedfrom communicating with the Base Node), then to third generation Nodes,etc.

At step 1607, one or more Nodes within the SVAN may be designated tocommunicate with a network access device extraneous to the SVAN. Forexample, a designated Node may aggregate data, such as an aggregation ofvalues for communication variables or sensor-generated data; andcommunicate the aggregated data to a destination outside of the SVAN(such as, via a cellular transmission or an IP Protocol transmission).

At step 1608, in some embodiments, a SVAN may be defined based upon anability of SVAN participant Nodes to communicate with each other via aprimary communication Modality. For example, a primary communicationmodality may include a Bluetooth modality, Wi-Fi, Wi-Fi RTT, sub-GHzradio transmission and the like. A secondary communication modality mayinclude IP transmission, a cellular transmission, sub-GHz communicationand the like.

At step 1609, some Nodes may be excluded, based upon an inclusion orexclusion criteria. For example, in some embodiments, only Nodes withunique IDs associated with sedans may be included in a SVAN, or onlyNodes with unique IDs associated with vehicles prepped for deployment(e.g., immediate rental) may be included in a SVAN, alternatively, Nodeswith IDs associated with vehicles recently returned and/or in need ofservice may be excluded from a SVAN.

At step 1610, communication variable values may be aggregated. Forexample, one or more Nodes or a controller may aggregate and store datathat is based upon, or quantifies, what transpires during a wirelesscommunication. Examples of data that quantifies, or is based upon, whattranspires during a wireless communication, may include, by way ofnon-limiting example, one or more of: a time of transmission, a time ofreceipt of a transmission, a phase angle of receipt of a transmission ofa single antenna, a respective phase angle of receipt of sametransmission by multiple antennas (which may include multiple antennasin one or more arrays of antennas). Other variables may include anamplitude of a received transmission, and a noise factor of a receivedtransmission.

At step 1611, a respective location of some or all of the Nodes in theSVAN may be generated, based upon the values for communication variablesthat are descriptive of communications with the respective nodes.

At step 1612, in some embodiments, an algorithm (such as those discussedherein) may be provided with values from the aggregated communicationvariable values to determine a location of a Node. Multiple sets ofvalues and/or multiple algorithms may be used to disparately determine aset of locations for a particular Node. The set of locations for theparticular Node may in turn by mathematically reconciled to determine abest location for the Node. For example, outlier sets of values may beset aside, included sets of values, and/or the set of locations for theparticular Node may be used to generate an average, a median, a weightedaverage, or other combined value.

At step 1613, a location of some or all Nodes in a SVAN may be plottedin a graphical representation. The location for a Node may be thelocations determined as described herein. In some embodiments, theunique IDs for plotted Nodes may be included in the graphicalrepresentation. Alternatively, or in addition to, the unique IDs, anannotation associated with a particular Node may be included in thegraphical representation. A graphical representation may include one orboth of two-dimensional and three-dimensional models of space occupiedby the SVAN. In some embodiments, these spatial models may be augmentedwith a time variable (e.g., by displaying a change in an area covered bya SVAN over time).

At step 1614, in some embodiments, a position of an Agent-supportedSmart Device may be determined relative to one or more of the Nodes in aSVAN. The Agent-supported Smart Device may be a smart phone carried by aperson or a smart device attached to a UAV or UGV. In some embodiments,the smart device will be programmed to communicate with a Base Node whenthe determines that it is within communication range with the Base Nodeusing a predetermined communication modality. For example, a GPSposition calculated by smart phone may indicate that the smart phone iswithin Bluetooth 5.1 range of a particular Base Node. The smart phone,acting as a Node, may then initiate Bluetooth 5.1 communication with theparticular Base Node.

At step 1615, using Orienteering methods, the SVAN may guide an Agentsupporting a Smart Device to a particular vehicle. For example, acustomer who has rented a car may be guided to that car via a graphicaluser interface on a smart phone. A controller may receive positioninformation of the rented car and the customer's smart phone and modifythe graphical user interface on the customer's smart phone to providedirections to the rented car. As the customer's smart phone begins theprocess by communicating with a first set of Nodes (that are withincommunication range of the customer's smart phone), and as the customertraverses a parking area (or areas proximate to the parking area), thecustomer's smart phone may transition to communicating with additionalNodes as those additional Nodes come with range of the smart phone. Agraphical user interface will be modified as the customer traverses theparking area to reflect in real time a relative of the customer and therented car (or other programmed destination, such as a rental car,office, or elevator).

At step 1616, in some embodiments, an angle of a viewing screen of thecustomer's smart phone relative to a ground plane may be determined asthe customer communicates with the SVAN. The angle of a viewing screenmay help determine if an image captured via operation of a smart phoneonboard-CCD image generator or other Image Capture Device is suitablefor inclusion in a graphical user interface. For example, most smartdevice-onboard CCD Image Capture Devices have a field of view that isgenerally perpendicular to a viewing screen of a smart phone.Consequently, a customer may hold up the customer's smart phone at anangle generally perpendicular to the ground plane and capture a view ofan area towards which the customer is walking.

At step 1617, a graphical user interface may be overlaid on top of animage captured by the CCD Image Capture Device in a positionperpendicular to the ground plane. Positions of Nodes within the fieldof view of the CCD device may be indicated in combination with the imagedata captured by the CCD device, based upon the verified position of theCCD device, an angle at which the CCD device is being supported and adirection of interest determined via automated Orienteering apparatusand methods.

Embodiments may include the positions of the Nodes within the field ofview of an Image Capture Device associated with the smart phone beingindicated as the vehicles with which they are associated, andinformation related to those vehicles. Information may include, forexample, an indication of which vehicle is being rented by a particularcustomer associated with the smart device; which vehicles need service;a vehicle type (compact, midsize, truck, specialty); which vehicles arerecently returned; which vehicles are ready to be rented, etc.

At step 1618, the graphical user interface may also include annotationsor other details as they relate to the Nodes and/or the associatedvehicles and/or aspects included in the field of view, such as a parkingrow number, an exit, an office, or other detail.

At step 1619, in another aspect, some embodiments may include an overlayof image data captured in a field of view with information descriptiveof or related to a Node with a position within the Field of View. Nodeinformation may include, for example, the unique ID associated with theNode, a Node model, battery charge remaining, signal strength, time oflast communication, details of data stored on the Node, amount ofstorage left in the Node, etc. In some embodiments, Nodes included in aGUI may be limited to those Nodes associated with vehicles and notdisplay Nodes deployed as reference position Nodes or associated withother items.

At step 1620, in still another aspect, in some embodiments, a Node fixedto or within a vehicle may continue to communicate after it exits aparking area. For example, if a Node is able to communicate with anotherNode, one or both of the Nodes external to the parking area may note aGPS location and store the GPS location in a manner associated with theNode-to-Node communication. If a Node is in a vehicle that is in motion,the Node may also note aspects of the travel of the vehicle in which theNode is located, such as, one or more of: speed, acceleration, vehiclediagnostics. Similarly, the Node may note a speed, acceleration andlocation of a Node with which it is communicating. All or somecommunication data generated as a result of the Node-to-Nodecommunication may be transmitted via a modality other than a modalityused for the Node-to-Node communication. For example, if Node-to-Nodecommunication is accomplished via a Bluetooth modality or a sub-GHzmodality, the information resulting from the Node-to-Node communicationmay be retransmitted via a cellular or IP modality to an off-SVANdestination. Off-SVAN destinations may include, for example, a server, acontroller or a smart device, in logical communication with the Internetor a cellular connection,

Referring now to FIG. 17 , method steps are illustrated for deploying aSVAN to manage activities, materials and people on a construction site.The construction site may include contract workers, tradesmen,management, guests, equipment, emerging building structure, undeployedmaterials and materials included in the structure, and the like.

At step 1701, a unique Node ID is associated with one or more of: onsiteAgent, material, equipment, structural aspect, or reference point (e.g.,pole placed onsite specifically to provide positional reference).

At step 1702, a first Node may be fixedly attached to, placed inside of,or otherwise co-located with one or more of: an onsite Agent, material,equipment, structural aspect, or a reference point.

At step 1703, reference point Nodes are located at strategic pointswithin or proximate to the Construction site. In some embodiments, thestrategic placements selected for reference point Nodes maybe based uponone or more of: a shape of the construction area; a wireless modalitydistance capability; a presence of obstacles within an area occupied bya SVAN; at ends of constructed elements on a construction site, and thelike.

At step 1704, one or more Nodes included in a SVAN may be designated asa Base Node. Base Nodes may be operative to perform functions notnecessarily performed by Nodes that are not Base Nodes. For example,Base Nodes may aggregate data over time, perform controller functions,transmit data via more than one wireless Modality, be powered byutility-based alternating current, or communicate via a hardwired medium(e.g., ethernet cabling).

At step 1705, one or more of the Nodes may communicate with other Nodes.Preferably, each Node will communicate with each other node within rangeof a communication modality. In some embodiments, a pattern of Nodecommunication may be followed.

At step 1706, in some embodiments, a pattern of communication maystagger a time of wireless communication in order to avoid interferenceof one communication by another communication. A pattern ofcommunication may therefore include a “cascade” or hierarchical tree ofwireless communication transmission and receipt. For example, a BaseNode may communicate first, followed by a first generation of Nodes thatreceive a communication from the Base Node, follow up by communicationfrom the first generation of Nodes with a second generation of Nodes(e.g., Nodes that are out of range or obstructed from communicating withthe Base Node), then to third generation Nodes, etc.

At step 1707, one or more Nodes within the SVAN may be designated tocommunicate with a network access device extraneous to the SVAN. Forexample, a designated Node may aggregate data, such as an aggregation ofvalues for communication variables or sensor-generated data; andcommunicate the aggregated data to a destination outside of the SVAN(such as via a cellular transmission or an IP transmission).

At step 1708, in some embodiments, a SVAN may be defined based upon anability of SVAN participant Nodes to communicate with each other via aprimary communication modality. For example, a primary communicationmodality may include a Bluetooth modality, Wi-Fi. Wi-Fi RTT, sub-GHzradio transmission and the like, and a secondary communication modalitymay include IP Protocol transmission, a cellular transmission, sub-GHzcommunication and the like.

At step 1709, some Nodes may be excluded based upon an inclusion orexclusion criteria. For example, in some embodiments, only Nodes withunique IDs associated with a particular type of equipment may beincluded in a SVAN, or only Nodes with unique IDs associated withmaterials prepped for deployment (e.g., immediate assembly into astructure) may be included in a SVAN. Alternatively, Nodes with IDsassociated with construction equipment recently returned or in need ofservice may be excluded from a SVAN.

At step 1710, communication variable values may be aggregated. Forexample, one or more Nodes or a controller may aggregate and store datathat is based upon, or quantifies, what transpires during a wirelesscommunication. Examples of data that quantifies, or is based upon, whattranspires dining a wireless communication, may include, by way ofnon-limiting example, one or more of: a time of transmission, a time ofreceipt of a transmission, a phase angle of receipt of a transmission ofa single antenna, a respective phase angle of receipt of sametransmission by multiple antennas (which may include multiple antennasin one or more arrays of antennas). Other variables may include anamplitude of a received transmission, and a noise factor of a receivedtransmission.

At step 1711, a respective location of some or all of the Nodes in theSVAN may be generated, based upon the values for communication variablesthat are descriptive of communications with the respective nodes.Methods and variables involved in determining a location for a Node arediscussed extensively herein.

At step 1712, in some embodiments, an algorithm (such as those discussedherein) may be provided with values from the aggregated communicationvariable values to determine a location of a Node. Multiple sets ofvalues and/or multiple algorithms may be used to disparately determine aset of locations for a particular Node. The set of locations for theparticular Node may in turn be mathematically reconciled to determine abest location for the Node. For example, outlier sets of values may beset aside. Included sets of values and/or the set of locations for theparticular Node may be used to generate an average, weighted average, orother combined value.

At step 1713, a location of some or all Nodes in a SVAN may be plottedin a graphical representation. The location for a Node may be thelocations determined as described herein. In some embodiments, theunique IDs for plotted Nodes may be included in the graphicalrepresentation. Alternatively, or in addition to, the unique IDs, anannotation associated with a particular Node may be included in thegraphical representation. A graphical representation may include one orboth of two-dimensional and three-dimensional models of space occupiedby the SVAN.

At step 1714, in some embodiments, a position of an Agent-supportedSmart Device may be determined relative to one or more of the Nodes in aSVAN. The Agent-supported Smart Device may be a smart phone carried by aperson or a smart device attached to a UAV or UGV. In some embodiments,the Smart Device will be programmed to communicate with a Base Node whenthe Stuart Device determines that it is within communication range withthe Base Node using a predetermined communication modality. For example,a GPS position calculated by a smart phone may indicate that the smartphone is within Bluetooth 5.1 range of a particular Base Node. The smartphone, acting as a Node, may then initiate Bluetooth 5.1 communicationwith the particular Base Node.

At step 1715, using Orienteering methods, the SVAN may guide an Agentsupporting a Smart Device to a particular piece of equipment, a set ofmaterials, a staging area, a drop off area, an office, or the like. Forexample, a worker who has placed a piece of equipment on a constructionlot may be guided to that equipment via a graphical user interface on asmart phone. A controller may receive position information of theequipment and the customer's smart phone and modify the graphical userinterface on the customer's smart phone to provide directions to theequipment. An Agent's Smart Device may begin the process bycommunicating with a first set of Nodes (that are within communicationrange of the customer's smart phone), and as the customer traverses aconstruction site (or areas proximate to the construction site), thecustomer's smart phone may transition to communicating with additionalNodes as those additional Nodes come with range of the smart phone. Agraphical user interface may be modified as the customer traverses theconstruction site to reflect in real time a relative location of thecustomer and the equipment.

In general, a user interface may be displayed upon a Smart Device, touchscreen or other human ascertainable mechanism. The interface may displaypositions of Nodes and/or associated Sensors, associated Structureaspects, communications paths between Nodes, communications interruptedby perceived obstructions, locations of items of interest, locations ofAgents, locations of non-Agent persons and the like.

At step 1716, in some embodiments, an angle of a viewing screen of thecustomer's smart phone relative to a ground plane may be determined asthe customer communicates with the SVAN. The angle of a viewing screenmay help determine if an image captured via operation of a smart phoneonboard CCD image generator or other Image Capture Device is suitablefor inclusion in a graphical user interface. For example, most smartdevice-onboard CCD Image Capture Devices have a field of view that isgenerally perpendicular to a viewing screen of a small phone.Consequently, a customer may hold up the customer's smart phone at anangle generally perpendicular to the ground plane and capture a view ofan area towards which the customer is walking.

At step 1717, a graphical user interface may be overlaid on top of animage captured by the CCD image capture device in a positionperpendicular to the ground plane. Positions of Nodes within the fieldof view of the CCD device may be indicated in combination with the imagedata captured by the CCD device, based upon the verified position of theCCD device, an angle at which the CCD device is being supported and adirection of interest determined via automated Orienteering apparatusand methods.

At step 1718, the graphical user interface may also include annotationsor other details as they relate to the Nodes and/or the associatedequipment, material, structural aspects, agents and/or aspects includedin the Field of View, such as a site topographic drawing references orother detail.

At step 1719, in another aspect, some embodiments may include an overlayof image data captured in a field of view with information descriptiveof or related to a Node with a position within the Field of View. Nodeinformation may include, for example the unique ID associated with theNode, a Node model, battery charge remaining, signal strength, time oflast communication, details of data stored on the Node, amount ofstorage left in the Node, etc. In some embodiments, Nodes included in aGUI may be limited to those Nodes associated with equipment, materials,agents, and the like. The GUI may not display Nodes deployed asreference position Nodes or associated with other items.

At step 1720, in some embodiments, Node information may be integratedinto Augmented Virtual Model (CAD), as well as any sensor co-locatedwith Nodes.

Referring now to FIG. 18 , method steps are illustrated for deploying aSVAN to quantify conditions in a parking area. The parking area mayinclude, for example, a garage or parking lot. Specific embodiments mayinclude one or more of: a rental car parking area; a commercial parkingarea; a residential parking area; a municipal parking area and the like.

At step 1801, a unique ID number is associated with a Node ID.

At step 1802, respective Nodes are placed within, or proximate to,multiple respective defined occupancy areas. The occupancy areas mayinclude, by way of non-limiting example, a hotel room and a healthcareprovider room.

At step 1803, a Sensor and/or Sensor assembly, such as a multi-sensormodule, is placed win logical communication with at least one Node thatis within or proximate to each disparate defined occupancy space. Insome embodiments, the strategic placement of Nodes maybe based upon oneor more of: a shape of the construction area; a wireless modalitydistance capability; a presence of obstacles within an area occupied bya SVAN; at ends of constructed elements on a construction site, and thelike.

At step 1804, one or more Nodes included in a SVAN may be designated asa Base Node. Base Nodes may be operative to perform functions notnecessarily performed by Nodes that are not Base Nodes. For example,Base Nodes may aggregate data over time, perform Controller functions,transmit data via more than one wireless Modality, be powered byutility-based A/C current, and/or communicate via a hardwired medium(e.g., ethernet cabling).

At step 1805, one or more of the Nodes may communicate with other Nodes.Preferably, each Node will communicate with each other Node within rangeof a communication modality. In some embodiments, a pattern of Nodecommunication may be followed (e.g., through the cascading processdescribed above).

At step 1806, in some embodiments, a pattern of communication maystagger a time of wireless communication in order to avoid interferenceof one communication by another communication. A pattern ofcommunication may therefore include a “cascade” or hierarchical tree ofwireless communication transmission and receipt. For example, a BaseNode may communicate first, followed by a first generation of Nodes thatreceive a communication from the Base Node, followed by communication bythe first generation of Nodes with a second generation of Nodes (e.g.,Nodes that are out of range or obstructed from communicating with theBase Node), then to third generation Nodes, etc.

At step 1807, one or more Nodes within the SVAN may be designated tocommunicate with a network access device extraneous to the SVAN. Forexample, a designated Node may aggregate data, such as an aggregation ofvalues for communication variables, Sensor-generated data; andcommunicate the aggregated data to a destination outside of the SVAN(such as, via a cellular transmission or an IP transmission).

At step 1808, in some embodiments, a SVAN may be defined based upon anability of SVAN participant Nodes to communicate with each other via aprimary communication modality. For example, a primary communicationmodality may include a Bluetooth modality, Wi-Fi, Wi-Fi RTT, sub-GHzradio transmission and the like, and a secondary communication modalitymay include IP transmission, a cellular transmission, sub-GHzcommunication and the like.

At step 1809, some Nodes may be excluded, based upon an inclusion orexclusion criteria. For example, in some embodiments, only Nodes withunique IDs associated with a particular occupant, or only Nodes withunique IDs associated with occupancy areas that Sensor readings indicateare vacant, may be included in a SVAN. Similarly, Nodes with IDsassociated with a group of persons or an item of equipment, as well asreference point position Nodes, may be included in inclusion orexclusion criteria.

At step 1810, communication variable values may be aggregated. Forexample, one or more Nodes or a controller may aggregate and store datathat is based upon, or quantifies, what transpires during a wirelesscommunication. Examples of data that quantifies, or is based upon, whattranspires during a wireless communication, may include, by way ofnon-limiting example, one or more of: a time of transmission, a time ofreceipt of a transmission, a phase angle of receipt of a transmission ofa single antenna, a respective phase angle of receipt of sametransmission by multiple antennas (which may include multiple antennasin one or more arrays of antennas). Other variables may include anamplitude of a received transmission, and a noise factor of a receivedtransmission. Data generated by Sensors associated with the respectiveNodes may also be aggregated.

At step 1811, a respective location of some, or all, of the Nodes in theSVAN may be generated, based upon the values for communication variablesthat are descriptive of communications with the respective nodes.Methods and variables involved in determining a location for a Node arediscussed extensively herein.

At step 1812, in some embodiments, an algorithm (such as those discussedherein) may be provided with values from the aggregated communicationvariable values to determine a location of a Node. Multiple sets ofvalues and/or multiple algorithms may be used to disparately determine aset of locations for a particular Node. The set of locations for theparticular Node may in turn by mathematically reconciled to determine abest location for the Node. For example, outlier sets of values may beset aside, included sets of values, and/or the set of locations for theparticular Node may be used to generate an average, a mean of othercombined value.

At step 1813, a location of some, or all, Nodes in a SVAN may be plottedin a graphical representation. The location for a Node may be thelocations determined as described herein. In some embodiments, theunique IDs for plotted Nodes may be included in the graphicalrepresentation. Alternatively, or in addition to, the unique IDs, anannotation associated with a particular Node may be included in thegraphical representation. A graphical representation may include one orboth of two-dimensional and three-dimensional models of space occupiedby the SVAN.

At step 1814, in some embodiments, a position of an Agent-supportedSmart Device may be determined relative to one or more of the Nodes in aSVAN. The Agent-supported Smart Device may be a smart phone carried by aperson, or a Smart Device attached to a UAV or UGV. In some embodiments,the Smart Device will be programmed to communicate with a Base Node whenthe Smart Device determines that it is within communication range withthe Base Node using a predetermined communication modality. For example,a GPS position calculated by a smart phone may indicate that the smartphone is within Bluetooth 5.1 range of a particular Base Node. The smartphone, acting as a Node may then initiate Bluetooth 5.1 communicationwith the particular Base Node.

At step 1815, using Orienteering methods, the SVAN may guide an Agentsupporting a smart device to a particular piece of occupancy area, suchas an occupancy area that Sensor data indicates is vacant or an areathat the Sensor data indicates is occupied.

In some embodiments, a controller may receive position information ofthe occupancy area and the Agent's smart phone and modify the graphicaluser interface on the Agent's smart phone to provide directions to aselected occupancy area. The Agent's smart phone may begin by beingguided via processing of values for variables of communications with afirst set of Nodes (what are within communication range of the Agent'ssmart phone), and as the Agent traverses a structure containing theoccupancy areas (or areas proximate to the occupancy area), the Agent'ssmart phone may transition to communicating with additional Nodes asthose additional Nodes come within range of the smart phone. A graphicaluser interface may be modified as the Agent traverses the c structurecontaining the occupancy areas to reflect in real time a relativelocation of the Agent and an occupancy area of interest

At step 1816, in some embodiments, an angle of a viewing screen of theAgent's smart phone relative to a ground plane may be determined as theAgent communicates with the SVAN. The angle of a viewing screen may helpdetermine if an image captured via operation of a smart phone onboardCCD image generator (e.g., charged coupled device camera) is suitablefor inclusion in a graphical user interface. For example, most smartdevice onboard CCD image capture devices have a field of view that isgenerally perpendicular to a viewing screen of a smart phone.Consequently, an Agent may hold up the Agent's smart phone at an anglegenerally perpendicular to the ground plane and capture a view of anarea towards which the Agent is walking.

At step 1817, a graphical user interface may be overlaid on top of animage captured by the CCD Image Capture Device in a positionperpendicular to the ground plane, and positions of Nodes within thefield of view of the CCD device may be indicated in combination with theimage data captured by the CCD device, based upon the verified positionof the CCD device, an angle at which the CCD device is being supportedand a direction of interest determined via automated Orienteeringapparatus and methods.

At step 1818, the graphical user interface may also include annotationsor other details as they relate to the Nodes and/or the associatedoccupancy areas and/or aspects included in the field of view, such as asite topographic drawing references or other detail.

At step 1819, in another aspect, some embodiments may include an overlayof image data captured in a field of view with information descriptiveof, or related to, a Node with a position within the field of view. Nodeinformation may include, for example, the unique ID associated with theNode, a Node model, battery charge remaining, signal strength, time oflast communication, details of data stored on the Node, amount ofstorage left in the Node, etc. In some embodiments, Nodes included in aGUI may be limited to those Nodes associated with a particular occupancyarea, or group of occupancy areas. The GUI may or may not, upondiscretion of a User or system manager, display Nodes deployed asreference position Nodes or associated with other items.

At step 1820, in some embodiments, Node information and occupancy areasmay be integrated into an Augmented Virtual Model (AVM) as well as datafrom any Sensor co-located with Nodes.

Referring now to FIG. 19 , method steps are illustrated for deploying aSVAN and displaying or communicating geolocated information. At step1901, the method may include associating a respective unique identifierfor each of at least a first Node; a second Node and a third Nodeincluded in an array of Nodes, wherein each of the first Node; secondNode and third Node comprises: a processor, a digital storage, acommunication module and an antenna.

At step 1902, the method may include designating a base position inrelation to the first Node.

At step 1903, the method may include wirelessly communicating betweenmultiple Nodes comprising at least the first Node the second Node, thethird Node and a fourth Node, wherein the fourth node includes an agentsupporting a smart device with a wireless communication capability whoenters a structure space comprising at least the first Node, the secondNode and the third Node, and wherein the fourth Node comprises anantenna array.

At step 1904, the method may include generating values for the firstNode; the second Node and the third Node, for communication variablesbased upon the wirelessly communicating between the first Node, thesecond Node and the third Node, wherein the communication variables mayinclude: one or more of a start time of a respective wirelesscommunication transmission (T1), a receipt time of the respectivewireless communication (T2), or a calculated transmission time. Thecommunication variables may also include one or more of a phasedifference of the respective wireless communication transmission betweena respective first antenna and a respective second antenna, or acalculated angle of arrival based upon the phase difference.

At step 1905, the method may include calculating relative positioncoordinates for the first Node, the second Node and the third node basedon the communication variables of step 1904.

At step 1906, the method may include generating values, for the fourthNode, for communication variables based upon the wirelesslycommunicating between the first Node, the second Node and the thirdNode, wherein the communication variables may include one or more of astart time of a respective wireless communication transmission (T1), areceipt time of the respective wireless communication (T2), or acalculated transmission time. The communication variables may alsoinclude one or more of a phase difference of the respective wirelesscommunication transmission between at least a respective first antennaand a respective second antenna within the antenna array of the smartdevice.

At step 1907, the method may include calculating a relative position anda relative orientation of the fourth node of the smart device based onthe communication variables of step 1906, wherein the relativeorientation determines a direction of interest of the user in thestructure space.

At step 1908, the method may include communicating information storedwithin the self-verifying array of nodes to the smart device, wherein aselection of data to transmit as the information utilizes the relativeorientation and relative position calculated for the fourth Node.

CONCLUSION

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. In some cases,the actions recited in the claims can be performed in a different orderand still achieve desirable results. In addition, the processes depictedin the accompanying figures do not necessarily require the particularorder show, or sequential order, to achieve desirable results. Incertain implementations, multitasking and parallel processing may beadvantageous. Nevertheless, it will be understood that variousmodifications may be made without departing from the spirit and scope ofthe claimed invention.

The headings used herein are for organizational purposes only and arenot meant to be used to limit the scope of the description or theclaims. As used throughout this application, the word “may” is used in apermissive sense (i.e., meaning having the potential to), rather thanthe mandatory sense (i.e., meaning must). Similarly, the words“include”, “including”, and “includes” mean including but not limitedto. To facilitate understanding, like reference numerals have been used,where possible, to designate like elements common to the figures.

The phrases “at least one”, “one or more”, and “and/or” are open-endedexpressions that are both conjunctive and disjunctive in operation. Forexample, each of the expressions “at least one of A, B and C”, “at leastone of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B,or C” and “A, B, and/or C” means A alone, B alone, C alone, A and Btogether, A and C together, B and C together, or A, B and C together.

The term “a” or “an” entity refers to one or more of that entity. Assuch, the terms “a” (or “an”), “one or more” and “at least one” can beused interchangeably herein. It is also to be noted the terms“comprising”, “including”, and “having” can be used interchangeably.

Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented incombination in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Similarly, while method steps may be depicted in the drawings in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order shown or in a sequentialorder, or that all illustrated operations be performed, to achievedesirable results.

What is claimed is:
 1. A method of communicating geolocated informationto a user, the method comprising: associating a respective uniqueidentifier for each of at least a first reference point; a secondreference point and a third reference point included in a virtualrepresentation of at least a portion of a building; designating a firstset of coordinates with the first reference point, second set ofcoordinates with the second reference point, and a third set ofcoordinates with the third reference point; associating a first visualidentifier comprising a first coded information with the first referencepoint, a second visual identifier comprising a second coded informationwith the second reference point, and a third visual identifiercomprising a third coded information with the third reference point;operating a smart device comprising an optical capture device to capturean image of each of the first visual identifier, the second visualidentifier, and the third visual identifier; determining a distance ofthe smart device from each of the first visual identifier, the secondvisual identifier, and the third visual identifier; generating aposition of the smart device in the virtual representation of at least aportion of the building based upon the distance of the smart device fromeach of: the first visual identifier, the second visual identifier, andthe third visual identifier; aligning a magnetic directional indicatorin a direction of interest; and communicating a selection of data to thesmart device based upon the position of the smart device in the virtualrepresentation and the direction of interest.
 2. The method of claim 1additionally comprising the step of an agent supporting the smartdevice; and wherein the step of aligning the magnetic directionalindicator in the direction of interest comprises pointing a specifiedside of the smart device in the direction of interest.
 3. The method ofclaim 1 additionally comprising the step of an agent supporting thesmart device; and wherein the step of aligning the magnetic directionalindicator in the direction of interest comprises pointing a symboldisplayed on a user interface of the smart device.
 4. The method ofclaim 1 wherein the selection of data communicated to the smart deviceis displayed via a user interface on the smart device.
 5. The method ofclaim 4 wherein the agent interacts with the user interface to selectthe selection of data communicated.
 6. The method of claim 5 wherein thefirst visual identifier, the second visual identifier, and the thirdvisual identifier comprise a hash tag.
 7. The method of claim 6additionally comprising the step of operating the optical capture devicein a line of sight with the first visual identifier, the second visualidentifier, and the third visual identifier.
 8. The method of claim 7wherein the smart device further comprises an accelerometer and issupported by an agent, and the method further comprises the steps of:operating the accelerometer; and based upon the operation of theaccelerometer, changing a view of information in the virtualrepresentation.
 9. The method of claim 8 additionally comprising thestep of designating a base position as co-located with a position of thefirst visual identifier.
 10. The method of claim 8 additionallycomprising the step of designating a base position as offset from thefirst visual identifier, according to a set of offset coordinates. 11.The method of claim 10, additionally comprising the step of designatinga position of a sensor as co-located with one of: the base position, thefirst visual identifier, the second visual identifier, and the thirdvisual identifier.
 12. The method of claim 10, additionally comprisingthe step of designating a position of a sensor as a fourth set ofcoordinates.
 13. The method of claim 10 additionally comprising the stepof, with a sensor, generating experiential data quantifying a conditionwithin the building.
 14. The method of claim 13 with a controller,generating a graphical user interface comprising a location of thecondition within the building.
 15. The method of claim 14 wherein thesensor a biometric sensor quantifying variables attributable to a humanwithin the building.
 16. The method of claim 13 additionally comprisingthe steps of capturing image data in a field of view of the smartdevice; and generating a graphical user interface comprising the imagedata captured in the field of view and the experiential data quantifyingthe condition within the building.
 17. The method of claim 16additionally comprising the step of including in the user interface apart ordering function associated with the position of the smart device.18. The method of claim 16 additionally comprising the step of includingin the user interface a virtual demonstration of a part and proceduresfor replacing the part in equipment.
 19. The method of claim 7additionally comprising the step of determining aspects of the buildingviewable in a physical world and selecting the data communicated to thesmart device based upon the aspects of the building viewable in thephysical world.
 20. The method of claim 19 wherein the data communicatedto the smart device comprises one or more of: as built data,experiential data, and performance data relating to machinery.
 21. Themethod of claim 19 additionally comprising the steps of: indicatingcompletion of a work order; and including in the data communicated tothe smart device an annotation indicating the completion of the workorder.
 22. The method of claim 21 additionally comprising the step oftriggering a payment mechanism based upon the completion of the workorder.